[Beta - Completed] New Search Backend

scotty on 29/10/2013 - 16:00

Hi OzBargain! We have updated OzBargain's search backend with a new implementation last night (which might have caused this morning's outage, but was quickly fixed). Now both the new and old search engines are running in parallel when we try to iron out the bugs. If you would like to test out the new search backend, here are the steps:

  • Go to any search page
  • Click on "Advanced search" to open the field group
  • Click on "Switch to Search Beta" button

Note that it works only for logged in users. The new engine is a lot faster. Search performance has always been a problem on OzBargain, and now you should be able to do sub-second searches. It also indexes one or two character words. Here is an example — search Nexus 7

  • Old backend takes ~3.50 seconds, and the first result is actually not about Nexus 7, as "7" wasn't used for indexing.
  • New backend takes ~0.34 seconds, and the first result is relevant.

The new search also prioritise keywords in the order of title, post and comments + some adjustment for recency. Still a bit of tuning required I suspect. Please give it a try.

Our next step would be fixing the search page user interface, and also per-account preference to have result sorted by recency only and preference to remove expired deals. I'll be looking at that next month.

Site Discussion

Comments

  • ESEMCE on 29/10/2013 - 16:22
    +3

    per-account preference to have result sorted by recency only

    Looking forward to that feature.. Unless the relevancy algorithm improves significantly. Perhaps 'recency" (is that even a word) can be scaled higher in the relevancy algorithm?

    I almost always use the Recent Items search, and I'm certain most users are the same.

    • ESEMCE on 31/10/2013 - 15:04
      +1

      Bit of feedback Scotty..
      First search using the Beta came back with 100% relevant and much better time ordered results in excellent time.

  • ozhunter on 29/10/2013 - 16:33
    +6

    Our next step would be fixing the search page user interface, and also per-account preference to have result sorted by recency only

    Looking forward to this feature.

    • evol on 29/10/2013 - 19:32

      +1 long overdue…

  • drewbie on 30/10/2013 - 09:36

    Out of curiosity, what are you using for your search backend? Solr? ElasticSearch?

    • scotty on 30/10/2013 - 10:02

      Sphinx search. That has very small memory footprint (less than 3MB RSS on x86-64) and fast enough for the amount of searches we do every day (which is just around 6k-8k/day). Both Solr/Lucene and ElasticSearch are Java-based, which is one thing I'm trying to avoid :)

      • drewbie on 31/10/2013 - 09:50

        At $work we're a perl shop, but I have no problem running a pair of solr servers with generous RAM. We get a LOT of searches and solr really does a marvellous job. When I get the chance, I want to upgrade to the lastest 4.x series for the improved speed and geosearch capabilities.

        • scotty on 31/10/2013 - 10:42

          … generous RAM

          Sorry I am a scrooge, especially with my servers :)

  • scotty on 31/10/2013 - 15:36
    +1

    ~35 people using the beta search at the moment. I'm considering switching it over for everyone tomorrow morning if there's no issue.

    • ESEMCE on 31/10/2013 - 15:40

      Do it.

    • scotty on 01/11/2013 - 00:24

      It's done. I'll be working on Search UI stuff later this month…

  • scotty on 21/11/2013 - 09:29
    +2
    Update — New Advanced Search

    There's a code update this morning and now we have included a new "Advanced Search", which should hopefully be easier to use. After search, you should see a grey-bordered box right underneath your search input box that currently have 3 selections:

    • Sort: Relevance — click on it to change sorting method. For example sort by post date.
    • All Nodes — click on it to limit the result to just classified, just forum topics, just deals or just deals excluding expired ones (which a lot of people are asking for)
    • All Categories — click on it to change the category to return. Really don't make sense if limiting search to classified or forum topics.

    If you are a logged in user, you'll see a 4th option — Save. Yes, it saves currently selected search options as default so your subsequent searches would use those options. I know some people would like to have "Sort: Post Date" and "Type: Deals (No Expired)" as their default.

    • cwongtech on 21/11/2013 - 12:31

      I don't know if this was supposed to happen - but the (New Comments) toolbar that would usually follow you and chill at the bottom of your screen + arrow buttons, it's gone.

      Also - when you click My Account - it used to show a list that contains Comments, profile, and stuff.
      There is no longer a list, not sure if the code broke some of the old stuff or it's supposed to happen?
      I can't vote on comments either :|

      EDIT: All the above just.. became alive! Problem's gone haha

  • scotty on 27/11/2013 - 08:34
    Update — Search Private Messages

    Now you can search not just the past deals & forum discussions, but also your private messages. If you have hundreds of PM in your inbox, it would be a very useful feature. To search private messages,

    1. Use the top-right search box in any of the private message pages, or
    2. Use this search link.

    Obviously it should only find the messages that you have participated in. The new messages are re-indexed once every hour.

Login or Join to leave a comment
Login Join