Apr 022012
 

Navigating the 1940’s census data website is painfully slow. Any requests submitted for data has the tendency to timeout and the choice to use Directly access and run census queriesAdobe to render imaging was horrible. Yep, Adobe was cool like 10 years ago. And… That felt quite good, now that we have gotten our typical griping out of the way, let’s have a quick chat about how you can use our API to access the newly released 1940’s census data. All you need to do is download our easy to use console application, supply a few command-line parameters and use it to render the data you are looking for quickly without having to access the website. All while leveraging  a more “direct” approach to census database queries. At this point, there are two different versions of the codebase. 1 will allow you to use command line for scripting, and the other option is a form based approach to retrieving the data. Both are relatively easy to use and full documentation is included in the install/zip.  Please be considerate of others who don’t already know about 1940cen.exe or would rather use the website. And if you don’t know what version you want to use – consider using the form first as its a tad bit more intuitive.

Here are some of the more power features you can leverage with 1940cen.exe

    • Submit multiple queries via one command line. Frankly, its quite annoying waiting (website) for your data to come back. The content rendering is complete bunk and timeouts appear to be inevitable. Which is one of the reasons we wrote the code. The other reason is we are dorks.
    • Search for a specific first name, last name, middle name
    • Eliminate the need render flash content and view as .tif by using the –E command followed by the image extension you would like to use. (see read me document for supported commands). We purposely eliminated any Acrobat extensions because we believe in a more “open sourced” approach to gathering and viewing public data.
    • Render maps and view data via Google maps API. Now this is probably one of the cooler functions, think of the possibilities! I know our researcher friends are drooling at the mouth :

How do I download 1940cen.exe?

quickly query data with our APIIts simple, just logon to our site, click on the “downloads” tab and select either the console or form version of 1940cen.exe.  The site is liable to get slammed with users and logons so be patient if it takes time to access the downloads. The zip file contains complete documentation on how to use the code, followed by several examples on how to use for non win32/64 operating systems. Should you prefer the non Windows OS, we have added support for Solaris, RedHat, and CentOS. Apples are to expensive so there is a very miniscule chance you will see any support there (unless you have a MS virtual machine of sorts).  There is even a mysql adapter for the ultra geeks that will allow you to cache your data for later use.

How should I best utilize this data, can you give me some example command lines approaches?

  • State: Use either -state or -s
  • County:  Use either -county or -c
  • City:  use either -city or -ct
  • Street:  use either -street or -s

Example query (enough already, I have the code and want the data).  Make sure you use % for any wildcard searches:

  • Command Line Examples:
    • directly query PA with a wildcard for 20%B and export as Google Map
      • 1940cen.exe get state=PA eid=20%b –M
    • general autonomous search (caution, there could be 100′s if not 1000′s of results).
      • 1940cen.exe state=PA county=Ful% city=t%
    • Search only street data if you don’t know any other information
      • 1940cen.exe street=belkin%avenue

Upon submission,  parameters, you can export the data as a Map, description, or census schedule simply by appending the correct command line flag. If you use the –M command line to export the map, we will export the rendering as a Google map.  Please let us know if you have any questions and please drop us a note if you have any difficulty or need any help.

Please be considerate to others (and our bandwidth) as well. Enjoy!

  6 Responses to “1940’s census data API”

  1. This is absolutely amazing. We had 10 researchers working to gather this data and just sent them all home. I was wondering if somebody would come up with an API to gather this data (it makes sense). The data retrieval engine was lightning fast. How did you do it? And no Flash? Awesome!!! We archived over 20k data entry points in less than 5 minutes which would have taken us weeks.

  2. Apparently “they” (census) are working on streamlining the data retrieval process. I’ve been trying like crazy to search for several family members for several hours. Many people want to see this data, and having an avenue such as this is great. Thanks for posting this your technology nerd friends. Consider my your 16th reader!.

  3. @P.Dwier and @A.Wolf, thanks for the notes! Yeah, we were surprised at how easy it was to run these queries and how quickly the data responded once eliminated the need for HTML. I can see this data being usefull to both the private and educational sectors where researchers can trend data and generate imaging based on the provided demographics! Good stuff!

    Also, if you dig the code or have adapted it to a specific use, please let us know how you are using 1940cen.exe.

  4. Thank you for posting this. We have used the form version of 1940 CEN. Having the opportunity to search with wildcards is great. We are amazed with how this application interprets handwriting with awesome accuracy.

  5. So it took me forever to find out how to logon or authenticate with my facebook account. Can you perhaps make it a little easier? I tried to post a link to your logon page but it appears as if the comment section does not allow for html links.

    This is a great tool for researchers! Thank you very much!

  6. There should have been a better approach to acquiring this data by the National Archives. Frankly, as a researcher I find it quite annoying one needs to access a sluggish website to singularly query the data. There is indeed a wealth of information available with this release and many people could benefit from it. This is like a horrible archive of scanned pages.

Sorry, the comment form is closed at this time.

Optimized by SEO Ultimate