Tag Archives: algorithms

micro-transit

Leave a reply

A number of public transportation agencies have been experimenting with micro-transit, where buses (or sometimes smaller vehicles) operate on-demand and are dispatched by algorithms. I like the idea – it seems like a possible way to provide service in low-density suburbs, unless we are going to start building differently. However, this op-ed from WHYY says it hasn’t gone well. Keep in mind the author is an advocate for transit riders and transit unions. It’s possible the person is cherry-picking examples or that the pilots in question for poorly implemented and managed.

On average, microtransit pilots across the U.S. have a ridership of zero to three riders per hour, with most pilots operating much closer to zero than three. For comparison, the Route 127, one of the most confusing and infrequent buses in SEPTA’s network, still moves an average of 13.9 passengers per revenue hour. When AC Transit in Oakland, Cal. replaced one of its  low-performing fixed-routes with microtransit, the per passenger subsidy more than doubled. And when Kansas City attempted microtransit, the ridership was so low that by the end of the pilot, they ended up paying $1,000 per passenger to operate the service.

WHYY

To be fair, this article is specifically arguing against implementation of this option by SEPTA (Southeast Pennsylvania Transportation Agency), which is not known for above-average implementation or management. The currently have an app which provides real-time data on bus and train arrival, but the data seems to be supplied by a random number generator. So I would not be too hopeful that they would be the first to pull this off successfully. Maybe they should just give everybody a pre-paid card to use Uber, or hire small-time taxi drivers who lost their life savings when that industry was upended a few years ago.

Snowden on Snowden

Leave a reply

Fresh Air called up Edward Snowden in his Moscow apartment and had an hour-long conversation with him. Among the interesting things he talked about is the idea that the combination of surveillance technology and cheap data storage means the NSA is essentially trying to collect all the world’s electronic communications, store them forever, and have them available both to search algorithms and human searchers. In other words, the idea is that an NSA staffer can just type in anyone’s name in the world and pull up any and all of the communications they have ever been involved in.

rain measurement using cameras

Leave a reply

This article is about estimating rainfall using ordinary surveillance camera footage and computer algorithms to process the videos. Measuring rainfall with physical rain gauges is subject to a lot of error, and so far the only real way to reduce the uncertainty is to add more gauges, which of course costs money. Radar can be used to improve our knowledge of what is going on in the spaces between rain gauges, but ultimately the radar-based estimates still end up being calibrated to the gauges. New methods to improve accuracy for a given gauge coverage, and/or reduce cost and gauge coverage while maintaining accuracy, would be welcome.

Advancing opportunistic sensing in hydrology: a novel approach to measuring rainfall with ordinary surveillance cameras

“Opportunistic sensing” represents an appealing idea for collecting unconventional data with broad spatial coverage and high resolution, but few studies have explored its feasibility in hydrology. This study develops a novel approach to measuring rainfall intensity in real‐world conditions based on videos acquired by ordinary surveillance cameras. The proposed approach employs a convex optimization algorithm to effectively decompose a rainy image into two layers: a pure rain‐streak layer and a rain‐free background layer, where the rain streaks represent the motion blur of falling raindrops. Then, it estimates the instantaneous rainfall intensity via geometrical optics and photographic analyses. We investigated the effectiveness and robustness of our approach through synthetic numerical experiments and field tests. The major findings are as follows. First, the decomposition‐based identification algorithm can effectively recognize rain streaks from complex backgrounds with many disturbances. Compared to existing algorithms that consider only the temporal changes in grayscale between frames, the new algorithm successfully prevents false identifications by considering the intrinsic visual properties of rain streaks. Second, the proposed approach demonstrates satisfactory estimation accuracy and is robust across a wide range of rainfall intensities. The proposed approach has a mean absolute percentage error of 21.8%, which is significantly lower than those of existing approaches reported in the literature even though our approach was applied to a more complicated scene acquired using a lower‐quality device. Overall, the proposed low‐cost, high‐accuracy approach to vision‐based rain gauging significantly enhances the possibility of using existing surveillance camera networks to perform opportunistic hydrology sensing.

MIT makes crazy AI on purpose

Leave a reply

A group at MIT showed an AI algorithm really disturbing pictures and then asked it what it saw in some inkblots.

When a “normal” algorithm generated by artificial intelligence is asked what it sees in an abstract shape it chooses something cheery: “A group of birds sitting on top of a tree branch.”

Norman sees a man being electrocuted.

And where “normal” AI sees a couple of people standing next to each other, Norman sees a man jumping from a window.

The psychopathic algorithm was created by a team at the Massachusetts Institute of Technology, as part of an experiment to see what training AI on data from “the dark corners of the net” would do to its world view.

The software was shown images of people dying in gruesome circumstances, culled from a group on the website Reddit.

Then the AI, which can interpret pictures and describe what it sees in text form, was shown inkblot drawings and asked what it saw in them.

Personally, I’m afraid of the people who spend their time seeking out photos of people dying violently and posting them on Reddit. As for the algorithm, I suppose it could be trained to identify and block violent images like the ones it has been shown, or kiddie porn, or ads targeting minors, etc. Or in the wrong hands, it could be used to block political speech or repress certain people or groups. Or the highest bidding company could get to use it to repress competitors’ ads.

Polarization, Partisanship and Junk News Consumption over Social Media in the US

Leave a reply

Maybe this is just the Brits picking on us. Or, maybe they are onto something.

Vidya Narayanan, Vlad Barash, John Kelly, Bence Kollanyi, Lisa-Maria Neudert, and Philip N. Howard. “Polarization, Partisanship and Junk News Consumption over Social Media in the US.” Data Memo 2018.1. Oxford, UK: Project on Computational Propaganda. comprop.oii.ox.ac.uk

What kinds of social media users read junk news? We examine the distribution of the most significant sources of junk news in the three months before President Donald Trump’s first State of the Union Address. Drawing on a list of sources that consistently publish political news and information that is extremist, sensationalist, conspiratorial, masked commentary, fake news and other forms of junk news, we find that the distribution of such content is unevenly spread across the ideological spectrum. We demonstrate that (1) on Twitter, a network of Trump supporters shares the widest range of known junk news sources and circulates more junk news than all the other groups put together; (2) on Facebook, extreme hard right pages—distinct from Republican pages—share the widest range of known junk news sources and circulate more junk news than all the other audiences put together; (3) on average, the audiences for junk news on Twitter share a wider range of known junk news sources than audiences on Facebook’s public pages.

I hadn’t heard the term computational propaganda before. Here is how they describe it:

The Computational Propaganda Research Project (COMPROP) investigates the interaction of algorithms, automation and politics. This work includes analysis of how tools like social media bots are used to manipulate public opinion by amplifying or repressing political content, disinformation, hate speech, and junk news.

We use perspectives from organizational sociology, human computer interaction, communication, information science, and political science to interpret and analyze the evidence we are gathering. Our project is based at the Oxford Internet Institute, University of Oxford.

So in other words, we are all being manipulated by some very old and tired ideas using powerful new technologies Hitler and Stalin could only have dreamed of.

Richard Berk

1 Reply

Here’s a Bloomberg article on Richard Berk, a statistician at the University of Pennsylvania whose algorithms are used for parole, probation, and sentencing decisions.

Risk scores, generated by algorithms, are an increasingly common factor in sentencing. Computers crunch data—arrests, type of crime committed, and demographic information—and a risk rating is generated. The idea is to create a guide that’s less likely to be subject to unconscious biases, the mood of a judge, or other human shortcomings. Similar tools are used to decide which blocks police officers should patrol, where to put inmates in prison, and who to let out on parole. Supporters of these tools claim they’ll help solve historical inequities, but their critics say they have the potential to aggravate them, by hiding old prejudices under the veneer of computerized precision. Some people see them as a sterilized version of what brought protesters into the streets at Black Lives Matter rallies…

 

The internet is telling you what you want to hear

Leave a reply

That’s right, the internet is telling you what you want to hear. In some cases, it really is government and corporate propaganda, known as “astroturfing“. This is the practice of creating a fake media buzz to give you the impression that there is grassroots support for something when there really isn’t:

Astroturfing is the attempt to create an impression of widespread grassroots support for a policy, individual, or product, where little such support exists. Multiple online identities and fake pressure groups are used to mislead the public into believing that the position of the astroturfer is the commonly held view.

Although usually associated with the internet, the practice has been widespread ever since newspaper editors first invented the letters page. Pick up any local paper around the time of an election and you will find multiple letters from “concerned residents of X” objecting to the disastrous policies of Y…

As reported by the Guardian, some big companies now use sophisticated “persona management software” to create armies of virtual astroturfers, complete with fake IP addresses, non-political interests and online histories. Authentic-looking profiles are generated automatically and developed for months or years before being brought into use for a political or corporate campaign. As the software improves, these astroturf armies will become increasingly difficult to spot, and the future of open debate online could become increasingly perilous.

The other thing going on is the “online filter bubble”, which is simply the idea that search and marketing algorithms are increasingly telling you what you want to hear. This makes sense in the logic of marketing, but is dangerous when you are trying to figure out what is going on in the world. From TED: