You are viewing ocschwar

entries friends calendar profile Previous Previous
ocschwar the unpronounced.
ocschwar
Things I want to play with: PANDAS. Time series data. RESTful interfaces. Machine learning algorithms.

To that end, I downloaded the WADL spec for data coming from ISO-NE.com. It's complete. There is no point meddling with WADL here, as all the service is read-only for yours truly. The important part of the WADL spec is the < grammars %gt; section, which is an XMLSchema.

That schema is complete, so I need something to parse it. I just tried generateDS.py, which almost does everything I need. The first failing is that it generates Python objects from the XML data I receive, and the objects all have the right attribute names and values, but it does not retain information about attribute types. I specifically need to know which attribute types are unique IDs (location IDs, mostly) so that I can make them columns in the PANDAS files I'm generating. The other failing is that it can only parse XML inputs. The JSON inputs which ISO-NE also provides, I cannot parse yet.
2 comments or Comment ?
ocschwar
1 comment or Comment ?
ocschwar
Comment ?
ocschwar
Over the last 20 years I spent many evenings at McIntyre & Moore's bookshop. There's just something about the stacks of used books in the carefully measured level of disorganization, that makes used books in general and M&M in particular a perfect place to unwind. And while I bought books there every once in a while, it was a fact that every visit there I would leave the store feeling sad that I could not buy all the titles I ogled there. There was the issue of time to read these books (as it stands right now if I don't figure out a good schedule for these things I will not be able to read what I have in my hosue right now, within my lifetime.) And the issue of space. And the issue of weight. But there was also the problem that while M&M had the best selection for good scholarly books, they were not priced to move. I could, and sometimes did, put my name in a bookmark, place it in a book in that store, and the bookmark would stay there for years. $30 dollars for a Springer Verlag book on physics. THat kind of thing.

A couple weeks ago I visited the new bookstore in Ball Square. It's a nice store, and the owner's a good guy. But the books, well, imagine an airport bookstore where the good books have all sold and not been replaced. And the ones that didn't sell never got remaindered. There was not a single title I could see there that I could find myself buying.

Don't it always seem to go..
That you don't know what you got till it's gone.
2 comments or Comment ?
ocschwar
2 comments or Comment ?
ocschwar
I got myself a Kindle Paperwhite specifically to stop my habit of printing scientific PDFs and lugging them around like dead weight literally for years as I read through them.

And quickly learned that while the Paperwhite is one of the best PDF readers on the market at the moment, that is not saying much. But every once in a while I develop an outright obsession, and this was one of those times. So some notes about this:

1. Elsevier is the spawn of Satan in so many ways, but they have one very nice feature: for any article they offer full text, they offer to generate MOBI files (as well as PDF and EPub). The service is flaky, but when it works, it just works, and the result is a scientific paper that displays properly.

2. Multicolumn PDFs suck. Single column PDFs are okay. And there is a Java app that converts the former into the latter: cut2col.

3. For IEEE papers, the HTML formatting is pretty nice. So:
a. Install Zotero, Calibre and pygnotero.
b. Take a snapshot of the HTML full text.
3. This script will then let you manipulate the snapshot into a MOBI:

from pygnotero import libzotero
import sys
import os
import subprocess

zotero_folder = '/home/omri/.mozilla/firefox/seb4oddc.default/zotero/'
z = libzotero.libzotero(zotero_folder)


The z.index structure holds all the metadata on your Zotero snapshots, with which you can manipulate the files into a MOBI. An example of what I did earlier:

z.update(True)
I =  z.index.items()
L = [ (i[1].key,i[1].title,i[1].authors) for i in I if i[1].key and  
     os.path.isfile(zotero_folder+'/storage/'+i[1].key+'/icp.mobi')]
print L
for i in L:
    a=  ["/usr/bin/ebook-meta" , "-t", i[1], 
           zotero_folder+'/storage/'+i[0]+'/icp.mobi']
    #p = subprocess.Popen(a,stdin=None,stderr=None,stdout=None)
    #p.wait()
    print a[-1]
    pass
Comment ?
ocschwar


Here's to you, JIRA.
Comment ?
ocschwar
Comment ?
ocschwar
This attempt to provide for your delectation an actual example of a minotaur dong was foiled by the failure of the Android LJ appp when uploading photos.
Comment ?
ocschwar
1 comment or Comment ?