If I never update this blog again, blame Sunlight Labs. I read their latest blog post and now I can’t… stop… playing… with… Scout.
Scout, at https://scout.sunlightfoundation.com/, is an alerts service which gives you updates on federal and state legislation, as well as speeches in Congress and Federal regulations. Federal legislation I’ve found all sorts of tools for, but when I was poking around for a place to get state legislative updates last year, I had a heck of a time — it was pretty much hit and miss and seemed to depend a lot on what state you’re in.
Scout starts out looking like a search engine so I did a simple search for “solar power”. The results page lets your break down your search results into several sections: bills in Congress, speeches, in Congress, state bills, and federal regulations. Choosing one of these allows you to do a little more delineation; for example, choosing to look at bills in Congress lets you choose what stage they’re at (passed, vetoed, etc.) and choosing state bills allows you to specify particular states.
Information in the search results is minimal; looking at solar energy bills in Montana provides brief information on the three bills that were returned, but additional information and the full text of the bill is no more than a couple clicks away. Similarly, searching for the number of times the word goofy has been used in congressional speeches (apparently former senator Byron Dorgan likes that word a lot) provides a brief context of the speech in the search results but the original speech is only a click away, with additional clicks to the source and the original GPO transcription with all its Green Acres references intact.
So the searching is good but the alerting is great. To use alerts you’ll need to have an account (it’s free) and if you want to get SMS alerts you’ll have to verify your phone number (Scout sends you a text and you enter the verification code from the text.) To set up alerts just do your searches. For every search result you’ll see a blue “Create Alert” button above the search results. Click that to save an alert.
All your alerts will be gathered in one spot, and you can edit them there to specify whether you want your alerts by text, e-mail, or not at all.
I immediately set up several alerts for state legislation; hopefully this’ll be easier for me to keep up with what’s going on in my state than what I’m using now, which is a couple of push notifications and lots of manual review. Thanks, Scout!
I got an e-mail yesterday from Charles R., who wanted help with a search he was trying to do. He wanted to be able to search just US state Web sites. He had tried to use site:gov as a search modifier but he reported it was missing too much.
As I was reading his e-mail I thought, “Doesn’t Google have a search for this?” Surprise! Uncle Sam Google Search, which searched only government Web sites, closed without fanfare over a year ago. No wonder Charles wasn’t having any luck.
So I thought about it for a few minutes. Charles knew how to limit his search to individual Web sites, but he didn’t know how to create a modifier for all possible state Web sites. And he’s right — it would be a heck of a job to do manually and would go past Google’s search limits. So I did it for him. Maybe you’ll find it useful too.
In my experience official state Web sites come in three flavors:
Name + gov (Alabama.gov)
Abbreviation + gov (Al.gov)
Abbreviation + us (Al.us)
So I made a list of the fifty states, with these three domain types for each state, and dumped it into a custom Google search engine that’s available at http://www.google.com/cse/home?cx=017167864583314760984:iecnygefhky.
Put in any keywords you want and your search results will be restricted to official state Web sites only.
That didn’t take too long to do, so I wondered if there were any other ways to slice the search results. And I came up with a couple. If you look at the search result page, you’ll see two tabs at the top that allow you to limit your search results to city or county Web sites only. These two restrictions just take advantage of the fact that city Web sites tend to use the pattern http://www.xx.ci.us and county sites tend to use the pattern http://www.xx.co.us . It’s not exaustively inclusive, but sample searches I ran brought me a lot of results.
Anyway, just something I quickly knocked together in order to help somebody out; it isn’t meant to take the place of Google Uncle Sam, just make it easier to search state Web sites. If you like it, if you can think of a way I can improve it, if you have a search refine problem you’re trying to solve, let me know. Thanks for sending the question Charles R! This was fun.
Just a quick look at this interesting Twitter directory. OH NOES, NOT ANOTHER TWITTER DIRECTORY! Yepper, here we go. Twitter States at http://www.twitterstates.com/ is a directory letting you browse Twitter users by US state.
Just pick a state and you’ll get a list of users from that state on the left and a bunch of weird eBay auctions on the right (I ignored those.) Listings include a full description of the Twitter-er, a information on how many friends and followers they have, and a calculated average of how often they tweet per day. Wow, some people tweet a lot…
A little additional information would have been nice, like special icons for verified accounts or “official” accounts (or maybe I just didn’t come across one?) Furthermore, this is only broken out by state, not city. I’m not sure how long it’s been around but it doesn’t have tons of registrants yet; potentially a good way to quickly find users in a certain state when news breaks. (Another would be to use Listorious or another Twitter list directory. It’s amazing how many state-based lists there are.)
You know, while I was going through this I was struck by how many other-than-human Twitter-ers I keep coming across. Maybe we need a new directory: nonhumantweeters.com . A place to put all those cats and birds and bridges and other things that aren’t biped-shaped but still manage to rock a tweetstream….
While I was writing the story on the new Merriam-Webster dictionary words I did several experiments with the new words. One of the words I experimented with — locavore — lead me to this great story about a database of over 15,000 growers around the country on a website built by Dan Sutton. His database, LocavoreNetwork, was launched in mid-May and is available at http://www.locavorenetwork.com/.
There’s actually a lot going on on this front page, with information on food safety, some forums (not very active), recommended books and resources for various kinds of gardening, and information on selecting and storing fruits and vegetables. What you’re looking for is the State Information tab, which will take you to http://www.locavorenetwork.com/content/state-information. You’ll still have to use the tab to choose the state whose farms you want to review.
I chose North Carolina. There’s actually two parts to this. First is an “availability guide” that shows you when various fruits and vegetables (from apples to watermelon) are in season. (Hey, where’s the zucchini? Where’s the okra?) The second is the “North Carolina Growers and Producers” section, which is a table (sortable by busines name or county) that show the business name, county, and what they grow, whether it be produce, meat, dairy, manufactured products, or wine (yes, there are wineries listed in this database.)
You can also search the database for various products. Each grower has its own page of details that you can get to by clicking on its name. The details vary a lot; Cane Creek Farm, for example, will give you everything including a full address, phone number, and Web site, while other listings like Green Toe Ground don’t even offer a phone number.
I honestly didn’t think I’d like this site with the way the information is divided by state. But the table is so well-organized and easy to search, and the availability charts so handy (if a bit incomplete) that I’m really impressed. If you’re at all interested in local farming check out this site. If you’re in the US, at least take a quick gander at the availability guide for your state.
This one’s been sitting in my queue for a while; I’m glad I’ve finally got the time to review it. I’m not even sure how long it’s been around. But it’s really good. The Library of Congress has launched Chronicling America: Historic American Newspapers, in beta. The site’s free and available at http://chroniclingamerica.loc.gov/ .
The site currently has keyword-searchable, scanned newspaper pages from 1880 to 1910 in nine states and the District of Columbia. Why so limited? Because it’s still being added to, and more content will be put in over time. There’s also a huge directory of information on newspapers published in the US from 1690 until today. Let’s look at at that first, then we’ll check out the scanned pages.
The front page has a link to the newspaper directory where you can browse by title, but skip that. Go right to the title search page. There you can do the GOOOOOD stuff: narrow down your search by state, county, or city; narrow it by span of time printed, find papers by ethnicity, frequency, or language, and of course search by keyword. And you’ll need to narrow down your search; just running a plain search for newspapers in New York found over 11000 results. *11000*. Yikes.
Results are listed alphabetically and there’s some data available; not enough, but some. Click on a paper name and you’ll get data like geographic coverage, dates of publication, language, frequency, publisher, etc. How much is available varies a lot; once I saw two papers with the same titles whose details varied slightly, but I didn’t get enough information on either one of them to tell them apart. This is really a jumpoff directory; find information on a paper here and use it to move on to searching richer sources.
The search of the newspaper pages, that’s completely different. It’s terrific! The way the search results are displayed is fantastic. But you’ll have to use the search page first: search by state or paper, by year or date range, and then use keywords which can include phrase or proximity search.Your search results include thumbnails of full newspaper pages! That sounds incredibly unwieldy but the places where your keyword appears are highlighted. When you choose a result you’ll get your page enlarged, again with your keyword highlighted. (I love those highlights — love love love — but I wish they were something besides rose color. Maybe highlighter yellow or nuclear green. If you’re searching for a keyword and it appears only once on a page, you’ll occasionally find yourself in a game of “hunt the highlight.”) You can get the text of the page (though it appears to be machine OCR’d and it looks pretty bad), a PDF, or you can download an image. Best of all, you can use a feature called “Draw Zoom Box,” outline a part of the page you want to enlarge, and immediately you’ll go to that area of the page — with the keyword highlighting intact. I was amazed at how smooth the zooming transition was. This is the most painless scanned-image newspaper searching I’ve done in a long time. In addition there are so many little extras — getting the pages in a variety of formats, several different levels of paper navigation even at the page-level viewing, and best of all, obvious permalinks to individual result pages. This project is going to be terrific. Between this and Wyoming’s project to digitize its newspapers, I may never read news from this century again. MORE PAPERS!
In an effort to get my information traps back up to snuff, I’m spending some more time messing around with Google News to get the best searches possible.
I keep information traps for both ResearchBuzz material and for my Day Job. For my Day Job, I like to keep track of how universities are using both textbooks and ebooks. While I do monitor the local universities’ newspapers directly, I want to be able to track trends on this usage across the country. This is where Google News comes in really handy.
As you probably know, Google Web search uses the “site:” syntax to restrict your searches to results that come from either one domain (example.com) or a set of “top level” domains (.com). The same syntax works for Google News. So if I want to restrict my news search results to just those that come from university sources, I can add site:edu to my search.
I want to monitor for news about electronic textbooks. My information trap would simply be
Of course, I might want to add variants, but that would be my start.
Does site: work with other domains? Sure; for government information you could use site:gov as a search modifier. Want military news? Try site:mil. You can also try those more unusual top-level domains like .biz, .info, and .tv, though it would be harder to narrow those down to one topic or one type of news.
Why not try country codes? I can hear the cool kids in the back saying if those unusual top-level domains should work, then why not use the country codes like .uk, .au, and .ca?
I don’t use country codes when I site search on Google News for two reasons. The first reason is that not all of a country’s media is on a domain that has a country code. For example, I really like The Irish Times. What’s the URL? http://www.irishtimes.com/. Not an .ie to be found. If I did a search for Irish news using site:ie then I would miss The Irish Times. And that would be bad.
The second reason I don’t use country codes with the “site:” syntax is that Google already has a syntax to let you search news by location. That syntax, surprisingly enough, is called “location:”!
“location:”, which is a syntax specific to Google News, allows you to restrict your search results to a particular country (or a particular state — more about that in a minute.) Use the name of the country with the syntax along with any keywords you want to use to restrict your search. For example:
That will find you stories containing the word “today” from the country of Malta. (And though Malta is a small country there are over 500 stories with the keyword “today”.)
I can’t guarantee that all countries will be represented in a Google News search because I haven’t tried all of them. But I found results from countries as small as Monaco:
So if you’ve got a trap and you want to aim it at a certain country, give that location syntax a try.
And once you’ve tried location: for countries, give it a whirl for US states! Yup, you can use location: with the two-letter USPS postal abbreviation for an American state, and you’ll get news from sources originating in that state. Let’s try this for Rhode Island:
As you can see, that search gets you news from sources like Pawtucket Times, Providence Journal, and Woonsocket Call. You can use this syntax with the 50 states, with the District of Columbia (location:dc) and even with at least one US-affiliated place, Guam (location:gu). (Strangely, Google News did not recognize either Puerto Rico or the US Virgin Islands as valid search locations.)
Knowing that we can search individual state news with the location: syntax, let’s go back to the site: syntax. Can you combine site: and location: syntax? Absolutely! Say I want to find college and university news in North Carolina. Nothing easier:
You don’t even have to use any search terms with that, though you will get a LOT of results. If I wanted to restrict my search for textbook or ebook information to one state, this would be the way to do it. Or if I wanted to focus my traps on one topic for one state, I could do it this way too:
basketball site:edu location:ca