api – Parerga und Paralipomena http://www.michelepasin.org/blog At the core of all well-founded belief lies belief that is unfounded - Wittgenstein Wed, 29 May 2019 14:38:55 +0000 en-US hourly 1 https://wordpress.org/?v=5.2.11 13825966 Introducing DimCli: a Python CLI for the Dimensions API http://www.michelepasin.org/blog/2019/05/24/introducing-dimcli-a-python-cli-for-dimensions-api/ Fri, 24 May 2019 11:10:15 +0000 http://www.michelepasin.org/blog/?p=3322 For the last couple of months I’ve been working on a new open source Python project. This is called DimCli  and it’s a library aimed at making it simpler to work with the Dimensions Analytics API.

The project is available on Github. In a nutshell, DimCli helps people becoming productive with the powerful scholarly analytics API from Dimensions. See the video below for a quick taster of the functionalities available.

Background

I recenlty joined the Dimensions team, so needed a way to get to grips with their feature-rich API (official docs). So, building DimCli has been a fun way for me to dig into the logic of the Dimensions Search Language (DSL).

Plus, this project gave me a chance to learn more about two awesome Python technologies: JupyterLab and its magic commands, as well as the Python Prompt Toolkit library.

Screenshot 2019-05-24 at 12.16.47.png

Features

In a nutshell, this is what DimCli has to offer:

  • It’s an interactive query console for the Dimensions Analytics API (ps: Dimensions is a world-class research-data platform including information about millions of documents like publications, patents, grants, clinical trials and policy documents.
  • It helps learning the Dimensions Search Language (DSL) thanks to a built-in autocomplete and documentation search mechanism.
  • It handles authentication transparently either via a global user-specific credentials file, or by passing credentials manually (e.g. when used within shared environments).
  • It allows to export results to CSV, JSON and pandas dataframes, hence making it easier to integrate with other data analysis tools.
  • It is compatible with Jupyter, e.g. it includes various magic commands that make it super simple to interrogate Dimensions (various examples here).
  • Feedback

    DimCli lives on Github, so for any feedback or bug reports, feel free to open an issue there.

    Screenshot 2019-05-23 at 18.06.32.png

     

    ]]>
    3322
    Getting hold of your Flickr collections with Python http://www.michelepasin.org/blog/2012/05/07/getting-hold-of-your-flickr-collections-with-python/ Mon, 07 May 2012 12:11:52 +0000 http://www.michelepasin.org/blog/?p=1889 Recently I’ve been a little disappointed with Flickr, the popular online photo-sharing service. Photos gone missing, entire albums disappeared. Not really what you’d like to happen to your photo collection, especially when it’s very large and therefore it’s difficult to be always on top of what’s there and what’s not.. Time to change strategy: use flickr for sharing and my local HD for backup!

    I emailed the customer service people at Flickr, they promptly replied that it wasn’t their fault but most likely a bug with other apps I had previously authorised to edit my Flickr collection (e.g. iPhoto or Aperture). Bad news: apparently whatever happened now what’s lost is lost forever. Not much to my consolation, the same happened to other people, for example check this post or this post to see alternative versions of the problem from 2010 and 2007.

    So I’ve suddenly realised the cloud isn’t that secure a place, as yet. It’s time to change strategy: use flickr for sharing and my local HD for backup!

    The good news is that if you know a little programming you can download your entire Flickr collection without having to pay a cent, for example by using Python. There are a few free libraries out there for accessing the Flickr APIs, such as flickrpy and FlickrAPI. They both require you to fiddle a little with the code (at the very least, get a personalised passkey from Flickr and add it to the python program) in order to get what you want.

    The one I’ve gone for instead is a little package called flickrtouchr, which is even easier to use. After downloading you just have to run it from the command line and it’ll begin browsing your whole Flickr collection and download pictures at the highest resolution available. I have more than 8000 photos, and it worked like a charm – beware though – it took more than 10 hours on my TalkTalk connection.

    Thanks Dan@hivelogic.com for writing this code – couldn’t be asking for more!

    [mac]@mike:~/Dropbox/code/python/_libs/dan-hivelogic-flickrtouchr-9ba645b>python flickrtouchr.py ~/Desktop/FlickrBackupFolder
    
    In order to allow FlickrTouchr to read your photos and favourites
    you need to allow the application. Please press return when you've
    granted access at the following url (which should have opened
    automatically).
    
    http://api.flickr.com/services/auth/?api_key=e2245325378b5675b4af4e8cdb0564716fa9bd&perms=read&frob=8856734hhgbbhsksd19443-caa77e89367asbbhfa2ba-600258&api_sig=a4aasdbbnb345c7fb46bdd33cfa65ec17bb32a
    
    Waiting for you to press return
    
    Egypt 1 ... in set ... Sharm el Sheik, Dec 2011
    Egypt 2 ... in set ... Sharm el Sheik, Dec 2011
    Egypt 3 ... in set ... Sharm el Sheik, Dec 2011
    Egypt 4 ... in set ... Sharm el Sheik, Dec 2011
    
    ..... etc….
    
    ]]>
    1889
    Hack4Europe! – Europeana hackathon roadshow, June 2011 http://www.michelepasin.org/blog/2011/05/23/hack4europe-europeana-hackathon-roadshow-june-2011/ Mon, 23 May 2011 21:36:14 +0000 http://www.michelepasin.org/blog/?p=1304 Europeana is a multilingual digital collection containing more than 15 millions resources that lets you explore Europe’s history from ancient times to the modern day. Europeana API services are web services allowing search and display of Europeana collections in your website and applications. The folks at Europeana have been actively promoting the experimentation with their APIs by organizing ‘hackathons’ – workshops for cultural informatics hackers where new ideas and discussed and implemented.

    Some examples of the outputs of the previous hackathon can be found here. Hack4Europe is the most recent of these dissemination activities:

    Hack4Europe! is a series of hack days organised by the Europeana Foundation and its partners Collections Trust, Museu Picasso, Poznan Supercomputing and Networking Center and Swedish National Heritage Board. The hackathon roadshow will be held simultaneously in 4 locations (London, Barcelona, Poznan and Stockholm) in the week 6 – 12 June and will provide an exciting environment to explore the potential of open cultural data for social and economic growth in Europe.

    Each hackathon will bring together up to 30 developers from the hosting country and the surrounding area. They will have access to the diverse and rich Europeana collections containing over 18 million records, Europeana Search API (incl. a test key and technical documentation) and Europeana Linked Open Data Pilot datasets which currently comprise about 3 million Europeana records available under a CC0 license.

    There are four hackathons coming up, so if you’re interested make sure you sign up quickly:

  • Hack4Europe! UK
    9 June 2011, London, hosted by Collections Trust
  • Hack4Europe! Spain
    8 – 9 June 2011, Barcelona, hosted by Museu Picasso
  • Hack4Europe! Poland
    7 – 8 June 2011, Poznan, hosted by Poznan Supercomputing and Networking Center and Kórnik Library of the Polish Academy of Sciences
  • Hack4Europe! Sweden
    10 – 11 June 2011, Stockholm, hosted by Swedish National Heritage Board
  •  

    ]]>
    1304