Showing posts with label Big data. Show all posts
Showing posts with label Big data. Show all posts

Tuesday, March 20, 2018

Monday, January 8, 2018

Frightening Tool Reveals How Much of Your Personal Info Facebook is Giving to Strangers

facebookA new online app allows users to input their Facebook information and see exactly what sort of information strangers, advertisers, and criminals can find on them.

Tuesday, October 24, 2017

First, ETFs; Now JPMorgan Incorporates A.I. Into FICC Business

Only yesterday, we commented on the launch of the world’s first A.I.-driven ETF “EquBot LLC, in partnership with ETF Managers Group (ETFMG) launched the world’s first ETF powered by artificial intelligence, the AI Powered Equity ETF (NYSE Arca: AIEQ). According to Business Wire, the new ETF uses ‘cognitive and big data processing abilities of IBM Watson™ to analyze U.S.-listed investment opportunities”.


A mere 24 hours later and JPM has joined the party as Bloomberg reports


At the world’s biggest debt dealer, traders can have trouble making sense of all the action as it happens. So JPMorgan Chase & Co. is bringing in artificial intelligence to give them a picture of the whole trading floor -- and even predict where markets are going. MSX, a data analytics and machine-learning program, is being deployed in the bank’s fixed-income sales and trading operations, New York-based JPMorgan said Monday in a statement. It will compile data from all desks and orders to give salespeople and traders a clearer picture in real time and help them anticipate market moves. Developed by London-based start-up Mosaic Smart Data, the program is already used in JPMorgan’s rates trading.



While the founders of EquBot claim that their proprietary A.I. can replace “an army of research analysts”, Mosaic’s CEO, Matthew Hodgson, had soothing words for the remaining human element in financial markets. He only wants to help salesmen and traders, not replace them.  “Automation of tasks doesn’t equal automation of jobs,’ Hodgson said in a phone interview. It will be another tool a fixed income salesperson can use to make his or her job more efficient and provide better service, he said. Hodgson is a former managing director at Deutsche Bank AG and has worked at Salomon Brothers, according to his LinkedIn profile.”


Mosaic uses the tagline "Your decision cockpit" for the MSX program and is targeting the FICC space:


“Financial institutions are facing a challenging period in FICC markets, largely as a result of the constraints of new regulatory initiatives, high fixed costs and a fragmented market structure. As the volume of data linked to trading activity and interactions with clients increases, the challenge to harness and analyse that data in real time becomes ever more critical. Data-driven banking no longer lies in the future. It is the here and now. Mosaic Smart Data® understands that the true value of data comes not only from the intrinsic individual data streams themselves, but also from the correlations and inferences that can be drawn from the aggregated data from each client. Our cutting edge technology addresses the challenges facing institutions trading in today’s FICC markets, including change management, productivity, efficiency, restructuring and the growing automation of trading processes.”



According to Bloomberg,


The software can identify patterns in data that JPMorgan already generates. It aims to predict client behavior and offer them ideas they’re more likely to be interested in. ‘It’s the ability to measure the business at the empirical, rather than the anecdotal level,’ Hodgson said.


 


‘If I’m really interested to know that, for instance, asset managers are active in a particular asset, I’d be very interested to know what maturity they’re contracting in.”



Mosaic joined JPM’s start up program “In Residence” for fintech companies last year and is the first company to pass out. The program gives start-ups access to JPM employees, systems and facilities and, presumably, some help with funding. Troy Rohrbaugh, JPM’s head of global macro trading stated “The Mosaic platform integrates securely with our existing technology infrastructure, and enables our teams to quickly make better informed decisions.”









Tuesday, September 12, 2017

2,000 Years Of Economic History (In One Chart)

Long before the invention of modern day maps or gunpowder, the planet’s major powers were already duking it out for economic and geopolitical supremacy.


Today’s chart tells that story in the simplest terms possible. As Visual Capitalist"s Jeff Desjardins notes, by showing the changing share of the global economy for each country from 1 AD until now, it compares economic productivity over a mind-boggling time period.


Originally published in a research letter by Michael Cembalest of JP Morgan, we’ve updated it based on the most recent data and projections from the IMF. If you like, you can still find the original chart (which goes to 2008) at The Atlantic. It’s also worth noting that the original source for all the data up until 2008 is from the late Angus Maddison, a famous economic historian that published estimates on population, GDP, and other figures going back to Roman times.





A MAJOR CAVEAT


If you looked at the chart in any depth, you probably noticed a big problem with it. The time periods between data points aren’t equal – in fact, they are not close at all.


The first gap on the x-axis is 1,000 years and the second is 500 years. Then, as we get closer to modernity, the chart uses mostly 10 year intervals. Changing the scale like this is a big data visualization “no no”, as rightly pointed out in a blog post by The Economist.


While we completely agree, we have a made an exception in this case. Why? Because getting good economic data from the early 20th century is already difficult enough – and so trying to find data in regular intervals before then seems like a fool’s errand. Likewise, a stacked bar chart with different years also doesn’t really do this story justice.


We encountered similar historical data issues in our Richest People of Human History graphic, and at the end of the day decided it was primarily for fun. Like today’s chart, it has its share of imperfections – but ultimately, it provides a great amount of context and serves as a conversation starter.


OUR INTERPRETATION


Caveats aside, there are many stories that materialize from this simple chart. They include the colossal impact of the Industrial Revolution on the West, as well as the momentum behind the re-emergence of Asia.


But there’s one other story that ties it all together: the exponential rate of human economic growth that occurred over the last century.



For thousands of years, economic progress was largely linear and linked to population growth. Without machines or technological innovations, one person could only produce so much with their time and resources.


More recently, innovations in technology and energy allowed the “hockey stick” effect to come into play.


It happened in Western Europe and North America first, and now it’s happening in other parts of the world. As this technological playing field evens, economies like China and India – traditionally some of the largest economies throughout history – are now making their big comeback.


Editor’s note: We have adjusted the main graphic as of Sep 10, 2017 to change the description of the chart. It now says “Share of GDP (World Powers)” instead of the previous “Share of world GDP”, which was technically an inaccurate description.

Monday, June 19, 2017

Records Of 198 Million US Voters "Accidentally" Exposed By RNC Contractor

While the Republican National Committee was allegedly busy evading Russian hacking attempts during the campaign, one of the contractors tasked with running its big-data operation apparently stored some of its most-sensitive files on an unsecured Amazon server that could be accessed by anyone who stumbled across the URL. As the Hill reports, Deep Root Analytics left a database containing 24 terabytes of data, including information about 198 million potential voters, or virtually the entire eligible population. The data included sensitive, but publicly available, information like voters’ addresses and phone numbers.


But more interesting than any personal information involved in the leak was the insight into Deep Root’s "big data" modeling tactics. The data included probabilities for individual voters’ positions on dozens of political issues, as well as estimates of how they voted in past elections. It also provided some insight into how big-data firms collaborate, as identifiers in the data suggest some of it was provided to Deep Root from TargetPoint and the Data Trust, two other big-data firms used by the Republicans.



Here’s the Hill:





For example, a 50-gigabyte file of "Post Elect 2016" information, last updated in mid-January, contained modeled data about a voter"s likely positions on 46 different issues ranging from "how likely it is the individual voted for Obama in 2012, whether they agree with the Trump foreign policy of "America First" and how likely they are to be concerned with auto manufacturing as an issue, among others."



That file appears in a folder titled "targetpoint," an apparent reference to another firm contracted by the RNC to crunch data. UpGuard speculates that the folder may imply that the firm TargetPoint compiled and shared the data with Deep Root. Another folder appears to reference Data Trust, another contracted firm.”


One UpGuard analyst who spoke with the Hill said he looked himself up in the data and that the estimated preferences were right on the money.


Gizmodo went into more detail about the exact nature of some of Deep Root’s models, including one that was used to predict voters’ opinions about the oil and gas industry in the US.





“One exposed folder is labeled “Exxon-Mobile” [sic] and contains spreadsheets apparently used to predict which voters support the oil and gas industry. Divided by state, the files include the voters’ names and addresses, along with a unique RNC identification number assigned to every US citizen registered to vote. Each row indicates where voters likely fall on issues of interest to ExxonMobil, the country’s biggest natural gas producer.”






The data evaluates, for example, whether or not a specific voter believes drilling for fossil fuels is vital to US security. It also predicts if the voter thinks the US should be moving away from fossil-fuel use. The ExxonMobil ‘national score’ document alone contains data on 182,746,897 Americans spread across 19 fields.”



Some of the data were apparently scraped from various subreddits, including, bizarrely, the banned subreddit “r/fatpeoplehate.” As Gizmodo noted, this is likely an attempt by Republicans to match social-media profiles with individual voters, a technique at Obama’s 2012 reelection campaign pioneered.






Some of the data included in Deep Root’s dataset veers into downright bizarre territory. A folder titled simply ‘reddit’ houses 170 GBs of data apparently scraped from several subreddits, including the controversial r/fatpeoplehate that was home to a community of people who posted pictures of people and mocked them for their weight before it was banned from Reddit’s platform in 2015. Other subreddits that appear to have been scraped by Deep Root or a partner organization focused on more benign topics, like mountain biking and the Spanish language.”






The Reddit data could’ve been used as training data for an artificial intelligence algorithm focused on natural language processing, or it might have been harvested as part of an effort to match up Reddit users with their voter registration records. During the 2012 election cycle, Barack Obama’s campaign data team relied on information gleaned from Facebook profiles and matched profiles to voter records.”



Reddit was an important locus of activity for Trump supporters during the campaign: it was the medium where they would create memes and share news stories that countered the mainstream media narrative. The subreddit currently boasts more than 440,000 members. Deep Root was paid handsomely for its efforts. The Hill, citing FEC data, said the firm collected $983,000 from the RNC between January 2015 and November 2016, according to Ad Age.



Though Deep Root has characterized the leak as an oversight, it still amounts to the largest leak of voter information ever.





“It dwarfs the second-largest exposure of voter information — 93.4 million records of Mexican citizens — by more than 100 million voters and tops the largest data breach of voter information — 55 million records of Philippine voters — by more than 140 million."



For its part, Deep Root said it was merely using the data to help target TV ad-buys. “Deep Root Analytics builds voter models to help enhance advertiser understanding of TV viewership. The data accessed was not built for or used by any specific client. It is our proprietary analysis to help inform local television ad buying."