'similar threads' function not functioning?

Here's an example...
This thread 'Creeping Charlie (Glechoma hederacea)'  https://permies.com/t/110127/kitchen/Creeping-Charlie-Glechoma-hederacea#901244
does not have any of the several 'creeping charlie' threads in it's 'similar threads' as are found in a search of titles.
I've been noticing this for awhile but it wasn't as noticeable as in this one...
https://permies.com/forums/jforum?module=search&action=search&forum_id=9&search_keywords=Creeping+Charlie&match_type=all&search_in=SUBJECT&forum=&forum=9&groupByTopic=true&sort_by=time&sort_dir=DESC&search_date=ALL&member_number=&member_first_name=&member_last_name=&member_match_type=memberPosted

I thought the similar ones were picked by subject or by title similarity?  Many of those listed for this thread don't seem to have anything in common with the topic.

(sorry I can't shorten my url on this tablet....)

I added a couple in manually.

I suspect, but don't know for certain, that similar threads system has some sort of scorecard it looks at when choosing threads.  If a thread has pictures, then it gets more score or something.  Likes might factor into it... or not.  It's a total mystery to me how it works, but there seems to be a pattern there.

I emailed devaka and he says he is looking into it.

This issue occurs when there is an auto-linked-keyword somewhere in the current page's text.

For example the thread contains the words, seed, medicinal, fence, perennial, fruit trees, experience, yard, and bees.

Joseph Lofthouse wrote:This issue occurs when there is an auto-linked-keyword somewhere in the current page's text.

For example the thread contains the words, seed, medicinal, fence, perennial, fruit trees, experience, yard, and bees.

That explains it!  and explains why it is only sometimes.

I wonder if there is some tech magic that could weigh the actual subject of the thread against those 'auto-linked' words?

EDIT...I just looked at the 'similar threads' for this thread...four 'creeping charlie' ones  

Judith Browning wrote:

Joseph Lofthouse wrote:This issue occurs when there is an auto-linked-keyword somewhere in the current page's text.

For example the thread contains the words, seed, medicinal, fence, perennial, fruit trees, experience, yard, and bees.

That explains it!  and explains why it is only sometimes.

I wonder if there is some tech magic that could weigh the actual subject of the thread against those 'auto-linked' words?

EDIT...I just looked at the 'similar threads' for this thread...four 'creeping charlie' ones  

Thank you, Judith and Joseph.  I think these two posts point out how irrelevant the current process is: it's not identifying similar threads, or rarely does so. The 'reactions' of the programming are, in effect, random and inane.  What kind of music do you get if you put a rabbit on a piano keyboard?  Posters include whatever words they want in a post, expressing themselves in the way they choose to in the moment. They're not at all thinking about how some programming will automatically associate words they've included in their post with some archived topic. I used to think the "similar threads" listed were annoying, but then went through a phase of getting a guffaw out of the absurdity.  If I look at the situation practically, my vote would be to get rid of that feature.

my vote would be to get rid of that feature.

When it works, it's quite helpful though and it does work well sometimes...seems like this is a more recent phenomenon?

I have confidence in the tech savey behind the scenes to come up with a fix when they've got the time.

Today I made some changes to the similar threads algorithm. For now, the new algorithm only works on new threads - threads created from now on.

Please keep an eye out on the similar thread lists of new threads. If they show an improvement, we can let the algorithm run on all threads.

Please keep an eye out on the similar thread lists of new threads. If they show an improvement, we can let the algorithm run on all threads.

Here's one posted since your changes. https://permies.com/t/110285/kitchen/Chicago-area-interested-receiving-raw#902240
 I don't see 'similarities' in the threads listed as 'similar threads'...not even any in the same forum as 'food choices'.
There are other topics about 'raw milk' though when I do a search.
https://permies.com/forums/jforum?module=search&action=search&forum_id=98&search_keywords=raw+milk&match_type=all&search_in=ALL&forum=98&groupByTopic=true&sort_by=time&sort_dir=DESC&search_date=ALL&member_number=&member_first_name=&member_last_name=&member_match_type=memberPosted

Same with this one https://permies.com/t/110284/Tire-repair-idea-dumb#902248
The similar threads don't seem similar except maybe the fact that they are mostly 'fixing' things? Three of them mention insulation in the title and all mention 'insulation' in the text, the OP uses the word twice but also uses the word tires or tire five times and there is no similar thread title with 'tire' in the title or in the text of the ones I looked at.

I think this is all  too subtle for me and I'm really just avoiding mowing the lawn

OK, finished the mowing and it's too hot for much else...so here's another new thread to test the algorithm .

https://permies.com/t/110286/Epic-Gardening-Hugelkultur-Beds#90226
It's in the hugelkultur forum...looks like one 'similar thread' has hugelkulture in the title but the others  listed don't seem related?

It seems like to be 'similar' they would at least share the same forum? or certain key words in the title?

I do see similar 'linked' words like 'gardening' in the text of the threads.

Thanks Judith! That's a wonderful analysis with a good bag of evidence :)

I'm going to do some further improvement to this stuff in about a couple of hours. Let's see how it works after that. I will update this thread when the new stuff is in place.

Joel Bercardin wrote:I used to think the "similar threads" listed were annoying, but then went through a phase of getting a guffaw out of the absurdity.  If I look at the situation practically, my vote would be to get rid of that feature.

Well, for one thing, Devaka is currently improving it, so let's not be hasty.

But the second thing to keep in mind is that the automated choice of similar threads that the software does is only the first part and (IMO) least important part of the feature.  We have staff tools for editing the similar threads list and curating it until it's as perfect as human endeavor can make it.  This is actually one of my favorite things to do, when I have time to do "Permies staff stuff."  It's extremely good for SEO and for making threads more valuable.  

That said, I don't actually think the existing algorithm that Devaka is improving right now was all that terrible.  It guesses a lot, it's pretty random, but it makes a startling number of good guesses, especially if you click the staff button that reveals fifty "more similar threads" to choose from so that you can uncheck any of the five default ones that are terrible and choose replacements from the list of fifty.  It's surprisingly rare that I can't improve the list of five by selecting from the list of fifty, although doing a keyword search will almost always be a still yet better way to get the "best five" threads listed.  

So, I'm looking forward to seeing the improvements, but the existing system is not terrible.  Except when the auto-linked keyword bug was messing it up.  

Another thing that may sometimes degrade results is the thread boost feature.  People can pay money to boost a thread in very large numbers of other threads; and so a thread where I once had five "perfect" links can, a year later, have some very random links in it due (I theorize) to thread boost.  This is not a bug, but a feature, in the sense that Paul making money is a feature.  But if there's any software effort to have thread boost happen in related/similar threads, I am not sure that it works well.  

I would very much miss this feature if it went away.  I think it adds an enormous amount of value to threads.  And if the links in a particular thread that the software chose aren't sufficiently similar, the solution is to manually choose better ones!

Dan Boone wrote:

Joel Bercardin wrote:I used to think the "similar threads" listed were annoying, but then went through a phase of getting a guffaw out of the absurdity.  If I look at the situation practically, my vote would be to get rid of that feature.

Well, for one thing, Devaka is currently improving it, so let's not be hasty.

Yeah, it's good to not be hasty.  And I always give our programmers & technicians full credit for the important work they do here.  I praise them for their efforts.  My feeling is that, for identifying similarities among topics described in the subtle English tongue, I suspect an algorhythm probably cannot compare with human consideration & understanding.

Devaka Cooray wrote:I'm going to do some further improvement to this stuff in about a couple of hours. Let's see how it works after that. I will update this thread when the new stuff is in place.

It took more time that I thought. The new stuff is in place. Please watch out for similar threads in new threads.

Dan Boone wrote:
But the second thing to keep in mind is that the automated choice of similar threads that the software does is only the first part and (IMO) least important part of the feature.  We have staff tools for editing the similar threads list and curating it until it's as perfect as human endeavor can make it.

This is the part of the process that I was mentioning is broken if there are auto-linked keywords in the currently displayed thread.

For example: I tried to add a similar threads link to the current thread.  https://permies.com/t/110127/kitchen/Creeping-Charlie-Glechoma-hederacea
The page that actually got added is: https://permies.com/t/72841/share-techie-stuff-buttons-push