planetwater

ground- water, geo- statistics, environmental- engineering, earth- science

Fun with Python: Script for Daily Newspaper

without comments

This post has to do neither with water nor with statistics. But because it’s been christmas break, I found some time to do some fun stuff with python:

I wrote a python script that downloads the issue of my choice of a pdf on my iPad. Partcularly, the script

  • lets me chose which issue of the paper (relative to the current day) I want to download
  • checks if that file might already exist on a relevant dropbox folder – checks if that file exists on the newspaper’s server (there is no paper on a holiday, f.ex) – if checks are ok, then the pdf is downloaded locally – saves the pdf in a selected folder within my dropbox – opens Goodreader, downloads the current file from dropbox (the proper url of the file needed to be determined) – deletes the downloaded pdf locally within pythonista

The relevant modules are the dropbox module of pythonista and the requests module to download the pdf with authentication.

The rationale behind attempting this was two fold:

  • using python on my iPad: I have downloaded pythonista a little while ago, but haven’t had time to play with it
  • since it has been possible, I have been subscribing to a German daily newspaper, which can be download as a pdf. I guess they are trying to foster the use of iPaper. I tried that for the last couple of weeks using iBooks on the iPad, and I don’t like it for two reasons:

    1. the iPaper files are larger than the pdfs, hence they take noticeably longer to load.
    2. I found it to be almost impossible to get the iPaper files out of iBooks on the iPad (to some storage place, for example).

The tipping bucket that led me to actually do this was that since a little while it is not possible anymore to download the pdfs directly from within Goodreader

Goodreader came out with an updated version two days ago. This makes the script even more powerful. This update solved issues related to the rendering of pdfs. I am not getting money from Goodreader, but I consider it to be the most useful software I have on my iPad. It particularly integrates awesomely with directories on an OSX Server.

Downloading the pdf

This is surprisingly easy using the requests package, that I just found out that it existed. Basically, downloading a pdf from a known url comes down to one line:

r = requests.get(url, auth=('XXX', 'YYY'))

Saving to Dropbox

This is step is a little bit more involved:

  • Before you start you have to register an ‘app’ in the developer part of dropbox. This will get you an APP_KEY and an APP_SECRET. In the python API you also need a token, which is from what I understand a string consisting of the name of your app in dropbox concatenated with '.token'.
  • When you’ve got that, I found some code on github that deals with the authentication process.
  • Finally, within the python module within pythonista, saving a file comes down to another one liner basically (with some preparations):

    sess = dropbox.session.DropboxSession(APP_KEY, APP_SECRET, ACCESS_TYPE) 
    client = dropbox.client.DropboxClient(sess) 
    f = open(filename) 
    response = client.put_file('/Apps/Pythonista/' + filename, f)
    

Open Issuess

  • the link between pythonista and good reader via dropbox works reasonably well. Would be nice to have a direct link (along the lines of ‘open in’).
  • it’s possible to create little home scree icons that call your code, also with command line arguments. As far as I can tell, you have to fix a certain value of the command line argument to one icon… that seems a bit against the flexibility of command line arguments
  • the time delta doesn’t treat boundaries of months correctly fixed by the first commit
  • not sure if raise Exception is really the way to go if a) the file exists already on dropbox or b) if there doesn’t exist a paper on the desired date

Conclusions

  • Pythonista offers a tool to achieve things that otherwise would be next to impossible on the iPad – at the same time, one has to wonder, why such things are forced to be impossible to begin with – writing code on the iPad without a dedicated external keyboard is… challenging. Despite the good editor of Pythonista.

The script is available on github.

Written by Claus

January 9th, 2014 at 4:23 pm

Posted in