Why are the forums hidden from Google?

[Spacebar]

I've always wondered why forum topics are all hidden from Google. If you Google something like "my headphones won't connect, moodeaudio forums", the top result is generally going to be another website. It's a bit annoying IMO because whether I'm searching a forum site or Reddit, I prefer to use Google, since built in search functions are generally inferior. 

Anyone know?

[kurt1970]

I've always wondered why forum topics are all hidden from Google. If you Google something like "my headphones won't connect, moodeaudio forums", the top result is generally going to be another website. It's a bit annoying IMO because whether I'm searching a forum site or Reddit, I prefer to use Google, since built in search functions are generally inferior. 

Anyone know?

Paid advertising. The more you pay, the higher in the ranking of the search results. Google is not a free lunch.

[Tim Curtis]

Thats prolly the reason. Our website moodeaudio.org never served Ads. Our Forum used to serve Ads but I found them really annoying on my phone so I finally just quit Google AdSense last year.

[Nutul]

If you Google something like "my headphones won't connect, moodeaudio forums", the top result is generally going to be another website.

From there it should not be that hard to FIRST go to the moodeaudio forum, THEN search there, uh?

[TheOldPresbyope]

I've always wondered why forum topics are all hidden from Google. If you Google something like "my headphones won't connect, moodeaudio forums", the top result is generally going to be another website. It's a bit annoying IMO because whether I'm searching a forum site or Reddit, I prefer to use Google, since built in search functions are generally inferior. 

Anyone know?

Uh, you do know Google search engine is not the only game in town, right?

Have you tried any of the leading (IMHO) alternatives such as DuckDuckGo, Brave, Microsoft Bing, Presearch? I just popped "my headphones won't connect, moodeaudio forum" into DuckDuckGo (the default setting in all my browsers) and got a reasonable set of answers, including a link to this thread.

Some of us are old enough to remember a time when Google didn't exist and AltaVista was the king of the Internet search world. We can even remember when the WorldWideWeb didn't exist and gopher was the answer to our prayers. Some of us are even old enough to...but now I'm sounding like my grandfather. Sigh.


Regards,
Kent

[challenge]

Moode forum and website do show up in the search results, but it seems it can't read or don't want to read the contents.
   

it shows a "Learn why" help thing, and if you're owner of the website it says that there's something not configured correctly in your robots.txt.
That points here:
https://support.google.com/webmasters/answer/7489871


Truth is that your robots.txt seems fine... there's something google doesn't like about the site. 
You are missing a sitemap for example.
Don't know if mybb now does it with or without additional extensions.
IIRC there was a google seo plugin for mybb to automate these things.
But you should have a sitemap in your robots.txt like this:

Code:

Sitemap: https://www.mywebsite.com/sitemap/
Sitemap: https://www.mywebsite.com/forum/sitemap-index.xml

and once it's accessible submit it through google webmaster tools and it'll index correctly the contents.
also give a look at what's showing up in google's Search Console, if it says something there.
SEO can be a pain, but this shouldn't be too hard.

Albeit I am trying to degoogle completely (as IMHO "Google is evil" and too powerful, personal thing), it still would be useful if the forum showed up correctly in the results instead of "No information is available for this page".
The issue is that diyaudio outranks this when you search something about moode and people are getting sent to that hundred pages long diyaudio thread (a mess).

[Tim Curtis]

Moode forum and website do show up in the search results, but it seems it can't read or don't want to read the contents.


it shows a "Learn why" help thing, and if you're owner of the website it says that there's something not configured correctly in your robots.txt.
That points here:
https://support.google.com/webmasters/answer/7489871


Truth is that your robots.txt seems fine... there's something google doesn't like about the site. 
You are missing a sitemap for example.
Don't know if mybb now does it with or without additional extensions.
IIRC there was a google seo plugin for mybb to automate these things.
But you should have a sitemap in your robots.txt like this:

Code:

Sitemap: https://www.mywebsite.com/sitemap/
Sitemap: https://www.mywebsite.com/forum/sitemap-index.xml

and once it's accessible submit it through google webmaster tools and it'll index correctly the contents.
also give a look at what's showing up in google's Search Console, if it says something there.
SEO can be a pain, but this shouldn't be too hard.

Albeit I am trying to degoogle completely (as IMHO "Google is evil" and too powerful, personal thing), it still would be useful if the forum showed up correctly in the results instead of "No information is available for this page".
The issue is that diyaudio outranks this when you search something about moode and people are getting sent to that hundred pages long diyaudio thread (a mess).

Good info.

I tried generating a sitemap using one of the online tools but there were > 500 pages to crawl and that requires a fee. If someone can get a full sitemap.xml file of http://moodeaudio.org I'll do the Google admin to make it active.

But what about bing, duckduckgo, etc search engines?

Btw, if I enter the OP's example search string in various engines the top hit is this thread...

[challenge]

I don't have recent experience with mybb, but by googling I think you should install an extension on this mybb forum that creates a sitemap file so that it updates the sitemap dynamically and automatically.
You could do it manually for small websites, but forums change, grow, let's say you add sections etc you should manually update the thing everytime...

A quick google in the mybb community website shows me there are two modules as option:

1. ABP sitemap https://community.mybb.com/mods.php?acti...w&pid=1279
   
2. Google SEO https://community.mybb.com/mods.php?action=view&pid=789 , github
   

pick one.
the first one does just the sitemap and easier to setup and seems more recently updated, 
the second one do a lot of more complex stuff, such as meta tags, redirects, error pages, has a github with more detail but more old etc

After having a publicly accessible sitemap file then you must link it in the robots.txt like I wrote in the previous post and google will find it next it will crawl. 
But I suggest also to submit the thing to google through their "google search console" tool so that happens faster.
a simple guide on that:
https://www.semrush.com/blog/submit-sitemap-to-google/

~~
Agree that the issue seems to happen only with google. They've become extremely picky through the years.
Happened to me too in past and had to submit the thing.
Provided there are no other issues like htaccess redirects, wrong file permissions, half hacked mods, or stuff like that.

[Tim Curtis]

I don't have recent experience with mybb, but by googling I think you should install an extension on this mybb forum that creates a sitemap file so that it updates the sitemap dynamically and automatically.
You could do it manually for small websites, but forums change, grow, let's say you add sections etc you should manually update the thing everytime...

A quick google in the mybb community website shows me there are two modules as option:

1. ABP sitemap https://community.mybb.com/mods.php?acti...w&pid=1279
   
2. Google SEO https://community.mybb.com/mods.php?action=view&pid=789 , github
   

pick one.
the first one does just the sitemap and easier to setup and seems more recently updated, 
the second one do a lot of more complex stuff, such as meta tags, redirects, error pages, has a github with more detail but more old etc

After having a publicly accessible sitemap file then you must link it in the robots.txt like I wrote in the previous post and google will find it next it will crawl. 
But I suggest also to submit the thing to google through their "google search console" tool so that happens faster.
a simple guide on that:
https://www.semrush.com/blog/submit-sitemap-to-google/

~~
Agree that the issue seems to happen only with google. They've become extremely picky through the years.
Happened to me too in past and had to submit the thing.
Provided there are no other issues like htaccess redirects, wrong file permissions, half hacked mods, or stuff like that.

Ok I'll look into it.

lol, maybe - "half hacked mods, or stuff like that" ;-)

[challenge]

yeah I really wanted to type "hard-coded" but you got it :-)

also there is a forked updated version of the second plugin ("google seo" one) updated for php8 that I missed before (don't know which php version you have)
it seems maintained by a mybb developer
Google SEO (Updated for PHP 8)
https://community.mybb.com/mods.php?acti...w&pid=1588
https://github.com/Sama34/MyBB-Google-SEO


many guides also talk about how bad is mybb for search engine optimization (missing meta tags etc...). Long story short it never was realeased in core as the forums usage dropped and everyone at the time moved to facebook and socials...

so maybe the "google seo" plugin is way more complete than the "ABP sitemap" one (that does just the sitemap), as it seems it does it all. 
It just need a bit more configuration that you can find in the github page...

do backup the files if it asks to manually hard-code something, as it could quickly get messy