Many organizations and people are just focused on technology during the implementation of Big Data and/or Data Analytics.  However, you should have the same predictive analytics result for the same algorithm no matter it is the programming code being written in SAS, R or Python.

The truth is that the proper methodology is far more important than which particular tool or technology being used.  I would like to share my experiences for some key elements for previous success on different analytic projects.

1. Locate the Business Pain-point(s)

A pain point is a specific problem that your business are experiences.  A business unit manager should be able to suggest which is the particular problem most difficult for their department.  Once the problem is located, it is important to collect relevant data and build corresponding analytics to tickle the pain point.  Also, it is a good exercise for team members to understand more in-depth of the business operation and fruitful for the whole organization.  Our analysis should be always focused on the key problems in order to archive the highest value of return.

2. Data Gap Analysis

For doing data analytic projects, a data scientist should have a detailed study on the data available before any analysis.  This is called data profiling including some basic statistics like key data distribution by histogram.  Meanwhile, it is possible for a piece of data being missed for a particular analysis.  If it happens, it is time to collect the missing data by introducing a new source or filled up by some manual input temporarily.

3. Start small and Implement by Iterations

Many Big Data / Data Science projects failed due to its complexity.  It is suggested to divide a complex problem into smaller pieces and it is much easier to tickle small problems one-by-one.  After the team is getting more familiar with handling data problems, it is possible for them to repeat their “small successful experience” to solve other issues by iterations.

4. Build data driven culture

Companies with strong data-driven cultures tend have top managers who set an expectation that forces decisions must be anchored in data.  So, it is a must for the data driven culture to be started from the top management.  Meanwhile, it is extremely important to train different staff to work with data by different roles like employing a data scientist, training existing staff to become citizen data scientists, etc.  For the implementation methodology, it is highly recommended to follow BADIR approach.

5. Automate the data applications

Previously, data analytics provide insights and leaving the decisions to be made by managers.  However, in order to response faster than your competitors, automated actions should be applied by data driven results.  For example, we have helped one of the largest battery manufacturers to build their own predictive analytic system on customer demand with proactive response.  It is now sending out alert email for clients with our predicted “running of stock” down to individual stores for retailed chains, supermarkets and convenient stores.

To sum up, data analytic is always a continuous journey.  In order to maintain the best efficiency of an organization, it is vital to review and refine the data analytics and data collection in a regular basis ensuring the continuous development and improvement with the business operations.

Witten by Samuel Sum, Vice President of AS / CDS / SDI (2020-07)