Liwaiwai Liwaiwai



Liwaiwai Liwaiwai
  • /
  • Artificial Intelligence
  • Machine Learning
  • Robotics
  • Engineering
    • Architecture
    • Design
    • Software
    • Hybrid Cloud
    • Data
  • About
  • Data
  • Machine Learning

What’s New In BigQuery ML: Non-linear Model Types And Model Export

  • November 25, 2020
  • relay

We launched BigQuery ML, an integrated part of Google Cloud’s BigQuery data warehouse, in 2018 as a SQL interface for training and using linear models. Many customers with a large amount of data in BigQuery started using BigQuery ML to remove the need for data ETL, since it brought ML directly to their stored data. Due to ease of explainability, linear models worked quite well for many of our customers.

However, as many Kaggle machine learning competitions have shown, some non-linear model types like XGBoost and AutoML Tables work really well on structured data. Recent advances in Explainable AI based on SHAP values have also enabled customers to better understand why a prediction was made by these non-linear models. Google Cloud AI Platform already provides the ability to train these non-linear models, and we have integrated with Cloud AI Platform to bring these capabilities to BigQuery. We have added the ability to train and use three new types of regression and classification models: boosted trees using XGBoost, AutoML tables, and DNNs using Tensorflow. The models trained in BigQuery ML can also be exported to deploy for online prediction on Cloud AI Platform or a customer’s own serving stack. Furthermore, we expanded the use cases to include recommendation systems, clustering, and time series forecasting.

We are announcing the general availability of the following: boosted trees using XGBoost, deep neural networks (DNNs) using Tensorflow, and model export for online prediction. Here are more details on each of them:

 

Boosted trees using XGBoost

You can train and use boosted tree models using the XGBoost library. Tree-based models capture feature non-linearity well, and XGBoost is one of the most popular libraries for building boosted tree models. These models have been shown to work very well on structured data in Kaggle competitions without being as complex and obscure as neural networks, since they let you inspect the set of decision trees to understand the models. This should be one of the first models you build for any problem. Get started with the documentation to understand how to use this model type.

 

Deep neural networks using TensorFlow

These are fully connected neural networks, of type DNNClassifier and DNNRegressor in TensorFlow. Using a DNN reduces the need for feature engineering, as the hidden layers capture a lot of feature interaction and transformations. However, the hyperparameters make a significant difference in performance, and understanding them requires more advanced data science skills. We suggest only experienced data scientists use this model type, and leverage a hyperparameter tuning service like Google Vizier to optimize the models. Get started with the documentation to understand how to use this model type.

 

Model export for online prediction

Once you have built a model in BigQuery ML, you can export it for online prediction or further editing and inspection using TensorFlow or XGBoost tools. You can export all models except time series models. All models except boosted tree are exported as TensorFlow SavedModel, which can be deployed for online prediction or even inspected or edited further using TensorFlow tools. Boosted tree models are exported in Booster format for online deployment and further editing or inspection. Get started with the documentation to understand how to export models and use them for online prediction.

We are building a set of notebooks for common patterns (use cases) for these models that we see in different industries. Check out all the tutorials and notebooks.

By Abhishek Kashyap, Product Manager
Source: Google Cloud Blog

relay

Related Topics
  • BigQuery ML
  • Explainable AI
  • Google Cloud
  • Kaggle
  • TensorFlow
  • TensorFlow SavedModel
You May Also Like
View Post
  • Data
  • Machine Learning

8 Best Human Behaviour Datasets For Machine Learning

  • January 30, 2023
View Post
  • Artificial Intelligence
  • Data
  • Machine Learning

Built With BigQuery: How To Accelerate Data-Centric AI Development With Google Cloud And Snorkel AI

  • January 29, 2023
View Post
  • Artificial Intelligence
  • Machine Learning

AI Might Be Seemingly Everywhere, But There Are Still Plenty Of Things It Can’t Do—for now

  • January 27, 2023
View Post
  • Data

Reading And Storing Data For Custom Model Training On Vertex AI

  • January 27, 2023
View Post
  • Data

Scaling Machine Learning Inference With NVIDIA Tensorrt And Google Dataflow

  • January 26, 2023
View Post
  • Machine Learning
  • Technology

GPT-3’s Next Mark: Diagnosing Alzheimer’s Through Speech

  • January 16, 2023
View Post
  • Data
  • Engineering
  • Practices

Building Out Your Support Insights Pipeline

  • January 13, 2023
View Post
  • Artificial Intelligence
  • Engineering
  • Machine Learning
  • Practices

Debunking 4 Common Myths About Machine Learning

  • January 12, 2023

Leave a Reply

Your email address will not be published. Required fields are marked *

Stay Connected!
LATEST
  • 1
    Microsoft‘s Big AI Ambitions Go Beyond Just OpenAI And ChatGPT
    • February 3, 2023
  • 2
    Deepfakes: Faces Created By AI Now Look More Real Than Genuine photos
    • February 3, 2023
  • 3
    GPT-3 In Your Pocket? Why Not!
    • February 3, 2023
  • 4
    Can AI Replace Cloud Architects?
    • February 2, 2023
  • 5
    Meet Aiko And Aiden: The World’s First AI Interns
    • February 2, 2023
  • 6
    Google Scrambles To Catch Up In The Wake Of OpenAI’s ChatGPT
    • January 31, 2023
  • 7
    9 Ways We Use AI In Our Products
    • January 31, 2023
  • 8
    Google Cloud Unveils New AI Tools for Retailers
    • January 31, 2023
  • 9
    7 Ways Google Is Using AI To Help Solve Society’s Challenges
    • January 30, 2023
  • 10
    The Ethics Of Machine Learning: Understanding The Role Of Developers And Designers
    • January 30, 2023

about
About
Hello World!

We are liwaiwai.com. Created by programmers for programmers.

Our site aims to provide materials, guides, programming how-tos, and resources relating to artificial intelligence, machine learning and the likes.

We would like to hear from you.

If you have any questions, enquiries or would like to sponsor content, kindly reach out to us at:

[email protected]

Live long & prosper!
Most Popular
  • 1
    8 Best Human Behaviour Datasets For Machine Learning
    • January 30, 2023
  • 2
    Built With BigQuery: How To Accelerate Data-Centric AI Development With Google Cloud And Snorkel AI
    • January 29, 2023
  • 3
    What Kind Of Future Will AI Bring Enterprise IT?
    • January 29, 2023
  • 4
    Prompt Engineering For ChatGPT And Generative AI
    • January 29, 2023
  • 5
    AI Might Be Seemingly Everywhere, But There Are Still Plenty Of Things It Can’t Do—for now
    • January 27, 2023
  • /
  • Artificial Intelligence
  • Machine Learning
  • Robotics
  • Engineering
  • About

Input your search keywords and press Enter.