Two Centuries of Population, Animated, using R

An interesting visualisation – history of a growing United States – mapping built using R. 

The animated map above shows population density by decade, going back to 1790 and up to recent estimates for 2015. The time in between each time period represents a smoothed transition. This is approximate, but it gives a better idea of how the distribution of population changed.

The Data used for this mapping is from the Census Bureau amd made better accessible by NHGIS.

What to expect in Apple’s big event tomorrow

It’s time for the biggest Tech event of the year – Apple’s product (hardware) launch event tomorrow. 

WWDC in June is when we find out about the greatest and the latest software Apple has built, but when it comes to how the software blends seamlessly with hardware, resulting in one of the best designed, engineered and built tech products on the planet, it’s the event in Fall, where they launch them. 

Key expectations are the next iPhone – iPhone 7, and the next big leap in wearables Apple Watch 2.

If there is one preview that you would want to read, about the event tomorrow, make it Jason Snell’s

The devil’s in the details, though. This event is Apple’s big chance to put all of its fall product offerings in context, to tell stories that explain why these products do what they do (or in some cases, don’t do what they don’t). This is product marketing at its highest level, and the way Apple introduces a product can be enlightening.

Apple getting rid of the headphone jack, what’s their take on wireless audio, the best camera on a smartphone getting even better (two lens camera), positioning of the Apple Watch – Jason has it all in his post. 

Gartner publishes Hype Cycle for Emerging Technologies 2016

Gartner has just released their annual Hype Cycle for Emerging Technologies, for 2016. 

16 new technologies added to the Hype Cycle this year, including blockchain, machine learning, general purpose machine intelligence, smart workspace for the first time.

Interestingly, 14 technologies were taken off the Hype Cycle this year including Hybrid Cloud Computing, Consumer 3D Printing, and Enterprise 3D Printing. 

Do checkout the report here. Definitely worth a read. 
Image source: Gartner

R moves up to 5th place in IEEE language rankings

IEEE has published its annual Top Computer programming languages rankings report. It starts with the line “C is No. 1, but big data is still the big winner”, indicating the rise of R, the defacto programming language used in Big Data analytics, including Cyber Security domain. 

I think this is an extraordinary result for a language which is domain-specific (big data and data science). If you compare R to the other four languages, which are general purpose languages (C, Java, Python amd C++) in Top 5, it’s a great feat, and is a clear indication of the adoption and heavy use and relevance of R in today’s Information Age where every device, system, or a “thing” (IoT) generates some form of data (logs). This also reflects the critical important of Data Science (where R is the defacto programming language used by Data Scientists), as a descipline today. 

Some interesting lines from the report:

Another language that has continued to move up the rankings since 2014 is R, now in fifth place. R has been lifted in our rankings by racking up more questions on Stack Overflow—about 46 percent more since 2014. But even more important to R’s rise is that it is increasingly mentioned in scholarly research papers. The Spectrumd efault ranking is heavily weighted toward data from IEEE Xplore, which indexes millions of scholarly articles, standards, and books in the IEEE database. In our 2015 ranking there were a mere 39 papers talking about the language, whereas this year we logged 244 papers.

R’s steady growth in this and numerous other surveys and rankings over time reflects the growing importance of Data Science applied using R. And application of Data Science concepts in Cyber security, especially in detecting  cyber attacks, is only becoming more and more relevant. 
Using conventional security monitoring tools which use rule based detection engines (yes they are called SIEM!), to detect cyber attacks, is not working anymore. Let’s face it; SIEM has come off age. Using Machine learning approach to detect cyber attacks, has become one of the most important developments in the cyber security domain in the last 10 years. And its relevance in today’s world, where there is surplus amounts of data (also called “Big Data”) being churned out by all forms of computer systems, is at its peak. And R is playing a very important role in helping Security Data Scientists build “algorithmic models” that can detect better cyber attacks 

So I am very excited and happy to see R’s popularity and adaption growing year on year. 

This is a core area of study I am currently focusing on, and I will be writing more about this here on my blog, in the coming months. 

Picture Courtesy: ieee.org

Verizon’s acquisition of Yahoo

TechCrunch just reported that Verizon has acquired Yahoo for $4.83 billion. 

This definitely is a shocker and I am sure many would agree with me. Not many of us were expecting Marrisa Mayer to call it a day by dropping the ball so soon. 

One of the most important companies of the first dot-com boom, Yahoo, has reached the end of its life as an independent company. This deal represents a stunnin decline for a company that was valued at more than $100 billion at its its peak in 2000. 

Marissa’s roots as an engineer at Google, definitely helped in improving the brand value with software programmers and technology users alike, and she did make an effort to beef up Yahoo’s technical talent. She instituted a regorous recruitment process and it worked hard at hiring computer scientists from some of the best universities. But there is little sign that these moves changed the culture at Yahoo or improved morale among the programmers working there. They always saw and projected themselves as a “media company” and not a “technology company”. I am not sure if it played out well for them, as its attempt to be a tech company and a media company at the same time, resulted in an organisation that was less than the sum of its parts. 

I strongly believe that one reason why Verizon was a strong contender was that they have done this before; Verizon acquired another struggling Internet company last year. Like AOL, Yahoo makes a lot of money by creating Internet  content and selling ads against it. So from Verizon’s perspective, this definitely looks like a logical step.

With respect to Mayer’s future at Yahoo, I am sure she is pursuing opportunities outside, as the statement that Yahoo released about this deal, “Yahoo will be integrated with AOL under Marni Walden, EVP and President of the Product Innovation and New Businesses organisation at Verizon”, makes it evident that Marissa Mayer’s future lies outside of Yahoo. 

I wish her all the best, and am sure she will build something very interesting soon in the tech business.



Picture courtesy: TechCrunch.com

On Tim Cook’s visit to India

This is the first time an Apple CEO has come to India. Steve Jobs had been here before, but that’s when he was soul searching and the India visit did play an important role in life thereon. 

Tim’s visit this week has been the most eventful and widely publicised, of all the large tech company CEO’s, in the recent past. I am sure this is going to have a huge positive impact on Apple’s market in India and a great benefit for Apple product lovers and customers here. 

Here is an interesting interview by The Hindu with Tim. I especially liked his reponse to a common and obvious question

Interviewer: Most of the billion people in India may not have heard about Apple. A few million would have heard and seen Apple products and only the minority few, who can afford it, would have actually used an Apple device. How would you as the CEO, explain what Apple is to this Indian audience?

Tim: Apple is about making the best products, we only create products that enrich peoples’ lives and in doing that we change the world in a positive way. That, in a simple way, is what Apple is about. Think of our products as tools to learn, teach; they empower people to do things they could not do otherwise. That’s our reason for being and that’s what drives us.

Development Centers in Bangalore and Hyderabad, and three Apple Stores supposedly in Bangalore, Delhi and Mumbai have been some of the interesting announcements. 

Interesting times ahead for Apple, consumers and entrepreneurs in India. 
Picture courtesy: dnaindia

Live transcription of OpenVis Conference

 

OpenVis Conference is a 2 day annual conference, held in Boston, about the practice of visualising data on the web. A must for all the Data Visualisation professionals amongst us.

This time, what is interesting is, they are Live streaming the Conference, in the form of Transcript, on their site, as shown below.Screen Shot 2016-04-25 at 9.44.24 PM

The conference is being held today (Apr 25) and tomorrow, and there are some really interesting Talks lined up. Some of these concepts have direct implication to Cyber/Information Security too.

I am hoping that the Presentations will be made available for people who couldn’t make it to the conference.

Interesting Data Science projects of 2015

Here is a list of some really interesting Data Science projects of 2015. Thanks to Jeff Leek from @simplystatistics for putting this together. 
Some of my picks from the list are:

* I’m excited about the new R Consortiumand the idea of having more organizations that support folks in the R community.

* Emma Pierson’s blog and writeups in various national level news outlets continue to impress. I thought this oneon changing the incentives for sexual assault surveys was particularly interesting/good.

* As usual Philip Guo was producing gold over on his blog. I appreciate this piece on twelve tips for data driven research.

* I am really excited about the new field of adaptive data analysis. Basically understanding how we can let people be “real data analysts” and still get reasonable estimates at the end of the day. This paper from Cynthia Dwork and co was one of the initial salvos that came out this year.

* Karl Broman’s post on why reproducibility is hard is a great introduction to the real issues in making data analyses reproducible.

* Datacamp incorporated Python into their platform. The idea of interactive education for R/Python/Data Science is a very cool one and has tons of potential.

Picture Courtesy: kdnuggets.com