Thank You!

Hello readers! I wanted to let you know that we will be having a new blog moderator at the Visual Insights blog. Please join me in welcoming Andrew Tom as the new moderator, who will be continuing the discussion on all things advanced visual analysis.

It has been great connecting with you all, and thank you for your continuing interest in Visual Insights.

– Elizabeth Hefner

The Impact of Big Data

The New York Times recently published an interesting article, The Age of Big Data, on the topic of the Big Data phenomenon and the opportunities that accompany the surge of information from new sources. In last week’s blog, we noted that a McKinsey Institute Study found that the demand for skilled data analysts will significantly increase, and the article discusses how this is just the beginning of the shift toward the need for more analytical understanding of Big Data.

“It’s a revolution,” says Gary King, director of Harvard’s Institute for Quantitative Social Science. “We’re really just getting under way. But the march of quantification, made possible by enormous new sources of data, will sweep through academia, business and government. There is no area that is going to be untouched.”

The article examines the many reasons and examples of why Big Data has an influence on fields such as business, government, and economics, and more importantly why decision-making will increasingly be based on data analysis. The article presents a compelling comparison of the impact of big data to the invention of the microscope. Erik Brynjolfsson, an economist at Massachusetts Institute of Technology’s Sloan School of Management, explains how the microscope was a dramatic change in measurement by enabling things to be viewed at the cellular level.

Data measurement, Professor Brynjolfsson explains, is the modern equivalent of the microscope. Google searches, Facebook posts and Twitter messages, for example, make it possible to measure behavior and sentiment in fine detail and as it happens.

Along with data measurement, being able to integrate, visualize, and analyze data will be the key to accurate and timely decision-making for organizations tackling Big Data. As we’ve previously discussed on this blog, Big Data has a significant impact on social network behaviors. The article presents a good example of the opportunities that Big Data presents in being able to effectively understand key patterns and relationships in increasing amounts of data from various sources and of different types.

Researchers can see patterns of influence and peaks in communication on a subject — by following trending hashtags on Twitter, for example. The online fishbowl is a window into the real-time behavior of huge numbers of people. “I look for hot spots in the data, an outbreak of activity that I need to understand,” says Jon Kleinberg, a professor at Cornell. “It’s something you can only do with Big Data.”

Overall, the article is a good discussion on the impact of Big Data, with real-world examples of how Big Data is influencing society and business and the effect it has on critical decision-making.

Building Integrated Insights from Big Data

As we’ve discussed on this blog in previous posts, the growth of Big Data – data that is increasing in volume, rapidly changing pace, and includes varieties of data such as structured and unstructured – presents organizations with new challenges as well as new opportunities. One of the challenges organizations face is that as the rate of data creation and consumption continues to accelerate, so will the demand for business data analysis professionals. As explained in the McKinsey Institute Study, the demand for analysts will largely surpass supply. However, in addition to the challenges, a key opportunity remains for organizations to better understand Big Data on another level – creating actionable, integrated insights across multiple data sources. Key drivers of the demand for leveraging Big Data to improve decision-making can be addressed through greater focus on self-service analysis powered with visualization, along with methods for building and analyzing insights from disparate data sources.

For instance, the importance of creating integrated insights can be seen in recent studies that have shown that activities in social networks involving key individuals correlate with information transfers and investment activities in security markets. For example, consider the activities of a portfolio manager. Portfolio managers place larger bets on firms they are connected to through their social network, and perform significantly better on these holdings and on holdings relative to their non-connected holdings. Not surprisingly, VC firms who are better networked also experience significantly better fund performance, as measured by the proportion of investments that are successfully exited through an IPO or a sale to another company.

Source: GigaOm

SNA is based in modeling relationships between groups of people. Social network analysis uses graph theoretic ideas and applies them with the premise that the structure of the graph can be used to understand and identify critical relationships and influential people. However, networks modeling real world relationships often have different characteristics compared to relationships used in SNA. For instance, to accurately “connect the dots” involving large quantities of diverse information and see patterns involving multiple aliases of people, different approaches are needed to handle the misleading, inaccurate, or incomplete information. One of the issues with traditional SNA is that the people and entities in a network are often not treated as adaptive and capable of taking action, learning, or altering their networks over time.

Recent advancements in network analysis involving complex network topologies with multiple relationships between nodes, network behavior that is based on uncertain information, and time-based change of networks, have enhanced the value of incorporating advanced network analysis techniques as a key part of an analytics toolset to aid in better understanding data relationships. More organizations are beginning to understand that with advanced visual analysis technology, they can build integrated insights across all of their available data, enabling them to better understand emerging opportunities and threats.

Using advanced analytics, businesses can study Big Data to understand the current state of their business and track customer behavior. Security and law-enforcement organizations can use advanced visual analysis to build integrated views of emerging threats from all available data sources to take timely action. The combination of advanced visualization techniques, together with social network analysis techniques, will help bridge the emerging gap between the vast amounts of available information in Big Data and the available resources to better understand them.

In the next series of blog contributions, we will continue to explore the incredible opportunities that Big Data represents, and new products and technologies that can contribute to better understanding and new insights.

New Blog Moderator and Direction

Hello Readers!

My name is Elizabeth Hefner, and as Stella Lau informed you, I’ll be the new blog moderator of Visual Insights. I want to thank all of you who have contributed to the blog already, and I’m looking forward to taking the blog in a new direction. As we all know, we are living in a data-driven world (see Accenture’s report on Big Data, and The Atlantic’s story on the “big data boom”). Not only are organizations bombarded with increasing volumes of data, but also data coming from new sources. As you can imagine, it is not enough to efficiently capture the data, but it must be better understood. Increasingly, many organizations are using visualization and analysis applications as part of their expanding portfolio of analysis software to discover new, valuable insights into their complex data.

Previously, Visual Insights focused more on data visualization. While data visualization plays a key part in insight discovery, I’ll be focusing this blog more on the topic of “advanced visual analysis”. If you are not familiar with the term, advanced visual analysis is what many analysts define as the new class of applications that tightly couple visualization and analysis, especially link analysis. These new applications go beyond pure visualization and traditional analytics, and reveal deeper, more complex relationships in data.

I’m excited to make this blog valuable to all data visualization and social network analysis enthusiasts, and I look forward to some insightful discussions! And remember, this community is an open forum for exchanging ideas, and comments and blog post submissions are greatly encouraged!

Elizabeth Hefner

Visual Insights Blog Moderator