CUNY Academic Commons

Hey Rafael et al,

I’ve been doing web scraping with the scrapy python library and doing some
scripting with pandas for cleaning datasets. I’d be really curious to
discuss with other people best practices and concepts for how they are
storing their scripts or setting up their python work environment.
Basically I’ve found myself with a bunch of…[Read more]

For more advanced control like dumping your scraped objects into dbs easily and sorting or granularity based on a text value I have been using the scrapy library. Their quickstart tutorial is pretty easy to follow and it’s best if you use xpath, but xpath is pretty simple to learn.

If anyone else is using scrapy and has questions/issues, I’m…[Read more]

Hey all – here’s a really interesting example of a 2nd tier newsroom taking a major major attempt at data vis. I think this highlights a lot of the struggles of mapping more than one story as described in class last week. But also it is striking to consider that data vis is so central to the final product of this 3 year investigation for a media…[Read more]

Shay- Good work on getting the linked selections to display like in your shoe viz.

Re:article – I never use location services, but when that article came out I used it to make the argument to all of my friends about why they shouldn’t grant location access to their apps.

For everyone: if you are a google maps user with location services turned…[Read more]

Hi all,

Here’s my “What can the cops learn about me from my phone?” post.

Excited to see yours.

Shay, I think you definitely need to share!! 😀

Annalisa

Extremely unfortunate that this event falls on the date of the Climate Strike!

US Climate Strike