The skills you need to succeed in Big Data
Firebrand Training
20 December, 2013
In the New Year, companies will have to make a decision, whether to hire new talent for Big Data or train existing data management talent with new skills. It will be a crucial decision, because Big Data is the next big deal.
Organisations, utilising big data differ from those with traditional data practices, because they:
- Pay attention to flows as opposed to stocks
- Rely on data scientists and product and process developers as opposed to data analysts
- Move analytics from IT into core business and operational functions
How does this manifest itself in business? Companies aim to measure customer sentiment or respond to breaks in train tracks in time to effect pre-emptive change. They need to analyse the data coming in from remote points as it flows in, not after it has been 'stocked' in a master database or migrated to a data warehouse.
They also need the statistical analysis skills to know which questions to ask of this data, and how to ask questions to arrive at new processes and even new products that the business sees commercial potential in. To get there, companies must have people possessing these technical skills as well as a strong business understanding.
In-demand skills to succeed
Nowadays, many businesses struggle to find suitable personnel, who tick all the boxes. Thus, the competition between Big Data professionals gets more intense, because those who really got the skills need to stand out. On a different note, Big Data in businesses doesn't run well without contributions from traditional data competencies; therefore the required skills are quite mixed.
“For instance, 59 per cent of companies responding to a 2012 survey conducted by analyst firm Information Difference said that their big data projects were 'highly linked' to their master data repositories. In many cases, master data (e.g. customer data, product data, and so on) was being used as 'vectors' into big data queries that began the process of probing piles of unstructured and semi-structured big data for clues on how customers react to certain offers, or how products were being accepted in certain markets, and so on.”
“In these cases, it was traditional master data that actually formed the core of what big data queries were constructed from — and so it was no surprise that 67 per cent of respondents in the same survey also said that master data was driving big data, rather than the other way around.” – ZDNet.com
Big Data skills vs. Traditional Skills
Big Data demands new programming and analytic skills, that today's typical data analysts lack. Most of these skills fall under the heading of 'data science'.
Key skills include:
- Strong Background in Mathematics
- Strong Background in Statistical Analysis
- Knowledge of Statistical Programming Languages
- Familiarity with Analytics Modelling Techniques
- Knowledge of Data Subject Matter
- Ability to Experiment with Data
Big Data also demands a new set of technical skills that aren't readily found today in many enterprise data centres. These skills include data architecting that includes the build-out of databases that span terabytes of data, being able to administer software frameworks like Hadoop, expertise in databases like noSQL, Cassandra or HBase; or in analytics programming languages and facilities like R or Pig.
But if these are some of the hard skills areas, Big Data also demands a set of soft skills that enterprise IT has customarily been short on. These include the ability of people to think across the organisation, to be aware of the ultimate needs of the business, to know which analytics questions to pose to get to those ultimate needs, and to measure and communicate results.
To learn more about the essential skills in Big Data, read the full article on ZDNet.com