The business case for leveraging Big Data is discussed extensively in conferences and publications worldwide – and it could not be clearer – it provides instant insight for better decision making. As a result, Big Data is on the mind of every business and technology leader from almost every industry sector imaginable, especially the public sector.
The Social Security Administration (SSA) uses big data to better identify suspected fraudulent claims and the Securities Exchange Commission (SEC) applies big data to identify nefarious trading activity. Many other Federal agencies are looking at ways to use Big Data to help execute their mission areas. In addition, a growing market of tools and techniques is now available to help Federal agencies effectively analyze large volumes of disparate, complex, and variable data in order to draw actionable conclusions.
Nevertheless, many of these same leaders are challenged in terms of successfully realizing the benefits Big Data has to offer, mainly because they:
Believe it is revolutionary and technology-focused, rather than an iterative and cyclical process.
Cannot determine the value of the data available to them. This challenge is multiplied when you consider that this data is consistently and rapidly expanding in terms of volume, variety, and velocity (in part, due to factors such as IoT, social media, video and audio files).
Are concerned about security and privacy issues.
Overcoming Big Data Challenges
Although these challenges can seem overwhelming, they can be overcome through a methodical process that is focused on improving your agency’s mission performance. They key is to adopt a use case-driven approach to determine how and when to begin your Big Data migration. Assuming your organization has already developed a Big Data strategy and governance framework, this iterative approach begins with your business (non-IT) stakeholders who support your organization’s primary or core functions. The objective is to use their institutional knowledge to define and prioritize a set of business needs/gaps which would improve their ability to perform their jobs more effectively. Once defined, the march towards Big Data begins in an iterative and phased manner.
Implementing Big Data in an Iterative and Phased Approach
Following the prioritized order, each business need should be decomposed into a use case (including items such as process flows, actors, and impacted IT systems). This decomposition will also help to facilitate the identification of all available data assets which touch upon the use case (private and public). In doing so, it is critical to brainstorm as broadly as possible. If a municipal government was trying to analyze how weather conditions impact downtown traffic patterns, they might use data from local weather reports, traffic camera feeds, social media reports, road condition reports, 911 calls reporting vehicular accidents, maintenance schedules, traffic signal times, etc. – not just the obvious traffic and weather reports. Each data asset should be assessed for quality (to determine if cleanup is required) and mapped to its primary data source.
Securing Big Data and Establishing Privacy Standards
Now that we have a handle on the data and the data sources that will be analyzed to support our use case, defining the proper security and privacy requirements for the Big Data analytics solution should be more straightforward (rather than defining a broad set of security controls for a Big Data environment in general). These requirements should be based upon the data assets with the highest sensitivity level, as well as the sensitivity levels of any analysis results that might be derived from various combinations of the data assets. The requirements will, in turn, enable the definition of the appropriate security controls that need to be integrated into any solution to reduce risk and prevent possible breaches.
Selecting the Right Big Data Solution
At this point, you are ready to start considering technology solutions to perform your Big Data analysis (such as the analytics, visualization, and warehousing, and tools). The technology you select, including infrastructure, core solutions, and any accelerators, should be based upon the information collected up to this point, including security requirements, data structures (structured vs. semi-structured vs. unstructured), and data volume.
Remember, no single technology is required for a Big Data Solution, rather it should be based on specific requirements. Data scientists can then use these technologies to develop algorithms to process the data and interpret the results. Once completed, you should move on to the next use case.
In summary, it is essential to remember that communication, change management, and governance are key to successfully derive any meaningful and usable results from Big Data. Other key success factors include:
Do not start with a technology focus. Instead, concentrate on business/mission requirements that cannot be addressed using traditional data analysis techniques.
Augment existing IT investments to address initial use cases first, then scale to support future deployments.
After the initial deployment, expand to adjacent use cases, building out a more robust and unified set of core technical capabilities.
These factors will ensure your agency adopts Big Data securely and effectively, achieving results at each iterative step and maximizing the use of your valuable resources.
Contact us today at info@eglobaltech.com, if you need help strategizing and implementing your Big Data project.
Copyright 2018 | EGlobalTech | All rights reserved.