In this article, I would like to share some technical staff rather than high level data science topic for managers. There are more and more organizations investing in Machine Learning (mostly TensorFlow) including the installation and configuration of new servers and GPU for the intensive computations.
Different hardware or cloud service will be discussed.
Development Workstation
For development workstation, it is important to categorize different types of workload like predictive analytic and Machine Learning (ML).
If there is something related to AI/ ML, it is recommended to have a workstation installed at least one powerful Graphic Processing Unit (GPU). As I am a poor data scientist, I have decided to use a refurbished gaming PC running Intel Xeon e5-2670v2 with the Nvidia GTX 2070 GPU.
However, it is better to have detailed comparison between the speed and performance between GPU with the URL below:
Deep Learning GPU Benchmarks – Tesla V100 vs RTX 2080 Ti vs GTX 1080 Ti vs Titan V: https://lambdalabs.com/blog/best-gpu-tensorflow-2080-ti-vs-v100-vs-titan-v-vs-1080-ti-benchmark/
Figure 1: best GPU for AI, Nvidia Titan V100
GPU Server
The technology giant – Google is using their own custom-made hardware to run TensorFlow for AI and machine learning. However, there is only Huawei providing AI specialized hardware in the public market. In order to have an alternative highly available, it is suggested to have a hardware box running a couple of fastest GPU units like NVIDIA 2080Ti (in gaming market) or NVIDIA Tesla / Titan (officially recommended) to do the intensive calculations.
Unfortunately, the data science team of my company is still using a box of containing 6 pieces of Nvidia GTX 1080Ti as a retired Ethereum miner. However, we are planning to upgrade the cluster with the memory size and speed concerns. It is suggested to have 32GB ram cards like Titan V100 for handling a huge Machine Learning / Deep Learning model under the TensorFlow framework.
Figure 2: our Ethereum miner with 6 GPU (1080 Ti)
Server (without GPU)
If your project is not related to TensorFlow nor similar framework(s), it is possible to use generic PC server(s) on Linux to run R / Python applications. For such environment, it is recommended to have at least single i7 or even dual Intel Xeon CPU. We don’t recommend AMD due to the potential compatibility problem of CPU instructions for programming library.
Cloud Consideration
The most flexible way of applying data science application is cloud service. It is possible to have software on cloud without any installation and configuration effort before doing your experiment. To take an example, it is possible to do predictive analysis directly with services like IBM Watsons Data Platform or TIBCO Data Science for AWS. Also, it is easy to run a Jupyter Notebook in AWS to do data analytics.
However, it is a good idea to have cloud infrastructure to get environment working before the completion. For the initial cost, it is quite low with only the usage costs. Nevertheless, the cost will grow significantly for long term extensive use by counting CPU and storage usage.
Another consideration is the security concerns on the cloud – especially public cloud. You may put testing data in the public cloud for data science development and testing. Finally, you have to prepare private cloud or your on-premise equipment for storing sensitive data like personal data, confidential commercial data, etc.
Conclusion
To conclude, the most practical way is to start your development at a cloud service with testing data on top of GPU. Once you have better understanding and some progress in development, then you may invest your own on-premise infrastructure and/or software to host your sensitive data to run data science model.
Samuel Sum
Data Science Evangelist (CDS, SDi)
Vice President (AS)