Learning Search

Smushy Search: A Tool for Topical Searching on Google

Every now and again I have to pause and marvel at the fact that Web directories as a category are pretty much defunct. There are specialized directories for specific categories, of course (while older directories like Jasmine and Starting Point have pivoted to business listings), but the idea of an Internet-wide searchable subject index dimmed considerably with the death of DMOZ. (Props to Curlie for soldiering on.)

This remains inexplicable to me and over the years I’ve come up with various schemes to remedy what’s a serious lack (I won’t bore you with them here.) At the same time I’ve mused a lot on the problem caused by a lack of Web directories – topic browsing is a pain!

Yes, you can go to Google and type cooking or beer or whatever and get search results that are useful. But you’re limited to whatever you find when you enter the topic through Google’s “front door.”  And those results are just one aspect of your topic (and depending on how popular that topic is, the results might be warped by SEO.)

How to get different results? Add more keywords, of course. And that takes us to the essential question of searching online: how do you ask about/think about/talk about what you don’t know?

Any keywords you can add to a search are predicated on your own knowledge/awareness. That makes topical browsing a challenge when your knowledge is limited (or worse, corrupted by mis/disinformation.) If you don’t have a subject index through which you are guided through a series of subtopics, how can you explore a topic that takes you beyond Google’s “front door” of results without relying on your knowledge of keywords (which may be nonexistent or incorrect?)

I tried to answer that question with Smushy Search. It’s available at https://researchbuzz.github.io/SmushySearch/ and does not require an API key or anything like that.

Screenshot from 2022-08-24 08-42-03

Making It Go

Here’s how Smushy Search works:

  • You enter the topic term for which you want to search (in the screenshot above it’s cooking.) Optionally you can add additional keywords to tilt your search in a certain direction (in the screenshot above it’s beginner.)
  • Choose if you want to filter out some common ecommerce results (removes some big ecommerce sites and tries to filter out other pages via search exclusions. Doesn’t work completely but it helps.)
  • Choose if you want to restrict your Google results to pages from edu and gov domains. Helps when you’re searching for something scholarly but will seriously narrow down your search results.
  • Finally, set the maximum frequency for the words that are going to augment the Google search. Word frequency is the number of times a word appears per million words of English text. The higher the frequency, the more common the word; light has a word frequency of a little over 173, while powdered clocks in at 2.35.

But where are the words coming from? That’s where the Datamuse API comes in.

Feeding Smushy Keywords With the Datamuse API

Once you’ve selected your words and options, Smushy Search sends your word to the Datamuse API. The API finds a set of adjectives related to your topic word and returns that to Smushy.  Here’s where the frequency option comes in. Adjectives that relate to your topic might relate to lots of other topics as well! By specifying a lower frequency, you can zero in on words that are both related to your topic and unlikely to bring you irrelevant results.

Smushy Search takes the array of words returned by the Datamuse API and filters it to eliminate any words that are more common than your frequency option. It then selects four of those words and groups them into two OR sets ( (wordone | wordtwo) (wordthree | wordfour) ) and adds them to your original keywords to make a Google query URL, which then opens in a new window. The query ends up looking like this:

(wordone | wordtwo) (wordthree | wordfour) topicword optionalword1 optionalword2

Smushy in Action

Let’s see how it works with the topic cooking and the tilt word beginner. When I hit the Smush button I get this page of Google results in a new tab:

Screenshot from 2022-08-24 08-37-39

Hmm, lots of cooking classes and easy recipes. (Do you see the query that Smushy used the Datamuse API to make?) Let’s do another search:

Screenshot from 2022-08-24 08-39-20

Now Smushy has taken us to a page of results about Asian cooking, with some healthy recipes thrown in. Let’s take one more spin:

Screenshot from 2022-08-24 08-38-53

Mmmmm… Creole food. Okra. Now I’m hungry. (Why do I always use food examples?)

Do you see how Smushy Search takes you past Google’s “front door” of searches, or even any keywords you might think of yourself? It’s like a Web randomizer, only it keeps you within a certain topic. It’s a fun toy for exploring and a bit of a timesink!

I shared Smushy Search with my Patreon supporters yesterday (without them I could not do this) and I had some feedback about expanding this to search other collections, like maybe Internet Archive or Google Books. What do you think?

Thanks for reading.

Leave a Reply