Talk 0: Meetup and Technology Updates
(Chris Fregly from PipelineIO)
• 6,400 Meetup Members!!
Thank You to Our New Sponsor, GoDaddy!!
PipelineIO Community Edition
• Hits 1,300+ GitHub Stars!
• 7,500+ DockerHub Downloads!!
• Global Reach in 50+ Countries on (Almost) Every Continent
• PipelineIO CLI
• Scikit-Learn Model Serving (joins Spark, TensorFlow, PMML)
• Latest Spark and TensorFlow Versions (All Optimizations)
• Full GPU Support on both AWS and Google Cloud
New PipelineIO **Premium Edition** Now Available!!
Contact: [email protected]
• 24x7 Global, On-Site Support
• Premium Training Included
• Spark + Tensorflow on GPUs
• Cloud + On-Premise Hosting
• Hybrid-Cloud High Availability
• Hybrid-Cloud Cost Optimizer
• Autoscaling + Spot Pricing
• Continuous Model Training
• Continuous Model Deploys
• Canary + Blue/Green Deploys
• Model A/B + Bandit Testing
• Model Server Circuit Breakers
• End-to-End Pipeline Monitoring
• Real-time Metrics Dashboards
Upcoming PipelineIO Workshop (20% Off)
RSVP Here: https://pipelineio-distributed-tensorflow-ai-gpu-workshop.eventbrite.com?discount=ADVANCEDSPARK20
• All GPU-based!!!
• Distributed TensorFlow AI Model Training and Serving
• Lots of Tensorboard Visualizations
• Heavy Emphasis on Post-Training Graph Optimizations
• A Bit of Spark and Scikit-Learn ML Model Training and Serving
• A Bit of CUDA Programming -- Why Not, Right?!
O'Reilly Online Training was a Huge Success:
High Performance Tensorflow in Production:
Develop hands-on experience optimizing and deploying TensorFlow Models
Talk 1: Deep Dive into Spark SQL's JsonPath Evaluator, Jackson Streaming, Scala Parser Combinators (Nathan Howell, Architect @ GoDaddy's Global Platform Team, Spark Contributor, Tensorflow Contributor)
We’ll explore how JsonPath expressions are parsed and evaluated one token at a time with Spark’s smallest interpreter. Learn about parser combinators in Scala, Jackson Streaming, and how compatibility with Hive's adhoc implementation was retained.
Nathan Howell is an architect in GoDaddy’s Global Platform team. He’s been contributing to Spark since 2014 and is an avid Tensorflow-user. Prior to joining GoDaddy he spent 15 years toiling away on large distributed machine learning systems at Microsoft, eBay and a few failed startups.
Talk 2: Reduced Precision (FP16, INT8) Inference on Convolutional Neural Networks with TensorRT and NVIDIA Pascal (Chris Gottbrath, Nvidia)
NVIDIA’s Pascal GPUs provide developers a platform for both training and deploying neural networks. In deployment GPUs allow lower latencies or servicing large inference workloads with a smaller set of accelerated nodes. One advanced technique to optimize throughput is to leverage the Pascal GPU family’s reduced precision instructions. I’ll show how you can start with a network trained in FP32 and deploy that same network with 16 bit or even 8 bit weights and activations using TensorRT. I’ll talk in some detail about the mechanics of converting a neural network and what kinds of performance and accuracy we are seeing on image net style networks.
I’ll end with a quick overview of how developers can deploy these DL networks as micro services using the GPU REST Engine.
Talk 3: High-Performance Tensorflow Serving across Hybrid Cloud and On-Premise + Request Batching + Model Post-Processing Optimizations (Chris Fregly from PipelineIO)
In this completely demo-based talk, Chris will demonstrate various techniques to post-process and optimize trained Tensorflow AI models to reduce deployment size and increase prediction performance.
First, we'll use various techniques such as 8-bit quantization, weight-rounding, and batch-normalization folding, we will simplify the path of forward propagation and prediction.
Last, we'll dive deep into Google's Tensorflow Graph Transform Tool to build custom model optimization functions.
Chris Fregly is a Research Engineer at PipelineIO - a Machine Learning and Artificial Intelligence Startup in San Francisco.
Chris is an Apache Spark Contributor, Netflix Open Source Committer, Founder of the Global Advanced Spark and TensorFlow Meetup, and Author of the O'Reilly Training and Video Series, "High Performance TensorFlow in Production"
Previously, Chris was a Distributed Systems Engineer at Netflix, a Data Solutions Engineer at Databricks, and a Founding Member of the IBM Spark Technology Center in San Francisco.