
{"id":7083,"date":"2022-09-28T16:22:18","date_gmt":"2022-09-28T16:22:18","guid":{"rendered":"https:\/\/rax-stg-wp-aws.com\/blogs\/bigquery-apis-reference\/"},"modified":"2026-11-03T19:27:48","modified_gmt":"2026-11-03T19:27:48","slug":"bigquery-apis-reference","status":"publish","type":"blogs","link":"https:\/\/www.rackspace.com\/de\/blogs\/bigquery-apis-reference\/","title":{"rendered":"BigQuery APIs Reference"},"content":{"rendered":"<p>Let us start with the definition first :<\/p>\n<p>BigQuery is an application that enables a fully-managed data warehouse that operates without any serverless architecture.The BigQuery enables the analysis of petabytes of data at once and makes the aspect of data analysis easier. BigQuery has features, which help in maintaining an impeccable stance on the analysis of data with the help of geospatial analysis, business intelligence and machine learning features.<\/p>\n<p>If anyone wants to know about it in detail, here is the&nbsp;<a href=\"https:\/\/cloud.google.com\/bigquery\/docs\">link<\/a>.<\/p>\n<p>But during my time in Rackspace, where I encountered some of the use cases in my project by which maybe I have forgotten the official \u201cdefinition\u201d, but I will remember the actual use of this<\/p>\n<p><strong>So where we can use BigQuery:<\/strong><\/p>\n<ul>\n<li>If in the legacy system, any query runs for a long and as the data grows over time so does the execution time of the query. So definitely it will enhance the performance.<\/li>\n<li>When you need to scale up your legacy server(SQL Server, Postgres, Oracle, etc.). so you can use BigQuery to reduce the load of your relational databases.<\/li>\n<\/ul>\n<p>Those two things I learned in terms of our project perspective and this list will continue.<\/p>\n<p><strong>Let&#8217;s see what options we can explore in terms of querying the data outside the GCP environment.<\/strong><br \/>\nThere are three methods to do that:<\/p>\n<ol>\n<li>REST API<\/li>\n<li>Python BigQuery Library<\/li>\n<li>ODBC and JDBC Driver for BigQuery<\/li>\n<\/ol>\n<p><strong>\u201cIntelligence is the ability to adapt to change.\u201d <\/strong>\u2013 <strong>Stephen Hawking<\/strong><\/p>\n<p><strong>REST API Method<\/strong><\/p>\n<p>This is a very popular method and the main advantage of this method is, you don\u2019t need to give admin\/owner level access.<\/p>\n<p>Let&#8217;s see it\u2019s implementation and testing. We need only a postman application for the testing.<\/p>\n<p><strong>Create OAuth Credentials<\/strong><\/p>\n<p>We will use Google uses OAuth 2.0 protocol for authentication and authorization. You will need a valid or approved key to access to the system and send requests to the BigQuery streaming endpoints. We need to avail the OAuth authorization framework 2.0 and retrieve the credentials of the client from the Google Cloud API.<\/p>\n<p>Login to the GCP console and after that navigate to APIs &amp; Services. Then enable the BigQuery API.<\/p>\n<p><img decoding=\"async\" src=\"\/wp-content\/uploads\/2026\/blog_article_images\/26249-body-Picture2_12.png\" alt=\"BigQuery Pic 2\" \/><\/p>\n<p>Then go to \u201cCredential\u201d option and Create an OAuth 2.0 Client IDs<\/p>\n<p><img decoding=\"async\" src=\"\/wp-content\/uploads\/2026\/blog_article_images\/26249-body-Picture2_11.png\" alt=\"Big Query Pic 2\" \/><\/p>\n<p>Please see the below configuration.<\/p>\n<p><img decoding=\"async\" src=\"\/wp-content\/uploads\/2026\/blog_article_images\/26249-body-Picture3_13.png\" alt=\"bigquery Pic 3\" \/><\/p>\n<p><a href=\"https:\/\/bigquery.googleapis.com\">Source URL<\/a><\/p>\n<p>This will create OAuth ID. Save these credentials (JSON) in your machine. We will use these credentials in Postman to generate a token for every request to the BigQuery streaming API.<\/p>\n<p>Now our task is done in GCP Console. Let&#8217;s open the Postman application<\/p>\n<p><strong>Generating Token in Postman<\/strong><\/p>\n<p>In the authorization section, click on \u201ctype\u201d and select OAuth 2.0<\/p>\n<p><img decoding=\"async\" src=\"\/wp-content\/uploads\/2026\/blog_article_images\/26249-body-Picture4_12.png\" alt=\"bigquery pic 4\" \/><\/p>\n<p>Now Configure the following parameters:&nbsp;<\/p>\n<ul>\n<li>Token Name: Any<\/li>\n<li>Grant Type: Authorization Code<\/li>\n<li>Callback URL: <a href=\"https:\/\/bigquery.googleapis.com\">https:\/\/bigquery.googleapis.com<\/a><\/li>\n<li>Auth URL: <a href=\"https:\/\/accounts.google.com\/o\/oauth2\/auth\">https:\/\/accounts.google.com\/o\/oauth2\/auth<\/a><\/li>\n<li>Access Token URL: <a href=\"https:\/\/oauth2.googleapis.com\/token\">https:\/\/oauth2.googleapis.com\/token<\/a><\/li>\n<li>Client ID: Copy Google cloud OAuth credentials from GCP console or from the json file downloaded from the OAuth console.<\/li>\n<li>Client Secret: Copy from Google cloud OAuth credentials console or from the json file downloaded from the OAuth console.<\/li>\n<li>Scope: https:\/\/www.googleapis.com\/auth\/bigquery<\/li>\n<li>Client Authentication: Send client credentials in body<\/li>\n<\/ul>\n<p><img decoding=\"async\" src=\"\/wp-content\/uploads\/2026\/blog_article_images\/26249-body-Picture5_12.png\" alt=\"bigquery pic 5\" \/><\/p>\n<p>Then click the \u201cGet New Access Token\u201d<\/p>\n<p>This will take you to Google cloud console page for login and authentication. Enter your credentials. An access token will be generated and you will be redirected to the Postman. Click on \u201cuse token\u201d<\/p>\n<p><img decoding=\"async\" src=\"\/wp-content\/uploads\/2026\/blog_article_images\/26249-body-Picture6_7.png\" alt=\"bigquery pic 6\" \/><\/p>\n<p>Navigate to \u201cHeaders\u201d section and enter \u201cContent-Type\u201d in Key and \u201capplication\/json\u201d in Value.<\/p>\n<p><img decoding=\"async\" src=\"\/wp-content\/uploads\/2026\/blog_article_images\/26249-body-Picture7_7.png\" alt=\"bigquery pic 7\" \/><\/p>\n<p>Navigate to \u201cBody\u201d section and enter the JSON request body. You will construct the body with the data to be sent to BigQuery.<\/p>\n<p><img decoding=\"async\" src=\"\/wp-content\/uploads\/2026\/blog_article_images\/26249-body-Picture8_6.png\" alt=\"bigquery pic 8\" \/><\/p>\n<p>Result in BigQuery<\/p>\n<p><img decoding=\"async\" src=\"\/wp-content\/uploads\/2026\/blog_article_images\/26249-body-Picture9_6.png\" alt=\"bigquery pic 9\" \/><\/p>\n<p><strong>Now Let\u2019s talk about method 2:<\/strong><\/p>\n<p><strong>Python BigQuery Library:<\/strong><\/p>\n<p><strong>Prerequisite \u2013 Before setting the environment you need to install the following package in the python environment.<\/strong><\/p>\n<p><em>pip install &#8211;upgrade google-cloud-bigquery<\/em><\/p>\n<p>The first step is to set up authentication to run client library.One way to do that is to create a service account and set an environment variable.<br \/>\n1.&nbsp;&nbsp; &nbsp;In the cloud console, navigate to the \u201cCreate service account\u201d page In the GCP Console option , go to the Create service account page.<br \/>\n2.&nbsp;&nbsp; &nbsp;Select the project on which you are working.<br \/>\n3.&nbsp;&nbsp; &nbsp;Enter a logical name in the Service account name field. Basically the Service account ID field is populated by Google Cloud Console.<br \/>\n4.&nbsp;&nbsp; &nbsp;There will a field named \u201cservice account description\u201d, enter your required description there.&nbsp;<br \/>\n5.&nbsp;&nbsp; &nbsp;For example, \u201cService account\u201d.<br \/>\n6.&nbsp;&nbsp; &nbsp;Then click Create and continue.<br \/>\n7.&nbsp;&nbsp; &nbsp;We need to provide access to the project. So, you need to grant the following role(s) to the mentioned \u201cservice account\u201d such as Project &gt; Owner.<br \/>\n8.&nbsp;&nbsp; &nbsp;There will be a field named \u201cselect a role\u201d, you can carry out your selection of roles from there.&nbsp;<br \/>\n9.&nbsp;&nbsp; &nbsp;For adding roles to the list, click on \u201cAdd another role\u201d.<br \/>\n10.&nbsp;&nbsp; &nbsp;Click \u201cDone\u201d button to finish creating the service account.<br \/>\n11.&nbsp;&nbsp; &nbsp;Do not hit the close button of your Browser\u2019s window, we will be needing this in the next step. Now create a \u201cService account key\u201d<\/p>\n<p><strong>Create a service account key<\/strong>:<\/p>\n<p>1.&nbsp;&nbsp; &nbsp;Now go to the GCP console and select email address that you registered for service account you created.&nbsp;<br \/>\n2.&nbsp;&nbsp; &nbsp;Click on \u201ckeys\u201d, then \u201cadd key\u201d and then select the \u201cCreate new key\u201d option.<br \/>\n3.&nbsp;&nbsp; &nbsp;Now select the \u201ccreate\u201d option and this will lead to the download of JSON key file in your computer or local machine.&nbsp;<br \/>\n4.&nbsp;&nbsp; &nbsp;Click Close.<\/p>\n<p><em>You can store this key in GCS Bucket or Passwordsafe Project. It will be needed for authentication.<\/em><\/p>\n<p><em>For Example :<\/em><\/p>\n<p><img decoding=\"async\" src=\"\/wp-content\/uploads\/2026\/blog_article_images\/26249-body-Picture10_5.png\" alt=\"bigquery pic 10\" \/><\/p>\n<p>I created one table in BigQuery for testing<\/p>\n<p><img decoding=\"async\" src=\"\/wp-content\/uploads\/2026\/blog_article_images\/26249-body-Picture11_5.png\" alt=\"bigquery pic 11\" \/><\/p>\n<p>Now please find the below python code for reference.<\/p>\n<p><img decoding=\"async\" src=\"\/wp-content\/uploads\/2026\/blog_article_images\/26249-body-Picture12_0.PNG\" alt=\"bigquery pic 12\" \/><\/p>\n<p>Result :<\/p>\n<p><img decoding=\"async\" src=\"\/wp-content\/uploads\/2026\/blog_article_images\/26249-body-Picture13_4.png\" alt=\"bigquery pic 13\" \/><\/p>\n<p><strong>Pros and Cons of Three Methods<\/strong><\/p>\n<p><strong>REST API<\/strong><\/p>\n<ul>\n<li><strong>Pros:<\/strong>&nbsp;For this method, no service account is needed, only need to create a bigquery client no need to share the JSON Key<\/li>\n<li><strong>Cons:<\/strong>&nbsp;Dynamically created tokens only last for 1 hour.<\/li>\n<\/ul>\n<p><strong>Python Client<\/strong><\/p>\n<ul>\n<li><strong>Pros:<\/strong>&nbsp;For any application that will run for a long and done CRUD Operation it can be used &nbsp;<\/li>\n<li><strong>Cons:<\/strong> For this service, the account is needed and \u201cOwner Access\u201d needs to be given there.<\/li>\n<\/ul>\n<p><strong>Conclusion<\/strong><\/p>\n<p>Those methods have their pros and cons, and it is perfectly synchronized with their use cases.<\/p>\n<p><em>JDBC and ODBC Driver<\/em><\/p>\n<p>This discussion is out of this scope, as this method is mainly used to connect any application like Tableau, QlikSense, etc. to BigQuery.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>BigQuery APIs Reference<\/p>\n","protected":false},"author":2,"featured_media":7085,"menu_order":0,"comment_status":"open","ping_status":"open","template":"","format":"standard","meta":{"content-type":"","nid":"26249","content_type":"blog","created_by":"10130","authored_on":"2023-11-16T17:22:18+00:00","field_article_industries":"","tag_category":"Technical","field_blog_author_text":"by Rahul Chatterjee, Site Reliability Engineer, Rackspace Technology","seo_description":"Explore data querying beyond GCP with three methods: REST API, Python BigQuery Library, and ODBC\/JDBC Driver for BigQuery.","field_featured_image":"","insights_image":"\/wp-content\/uploads\/2026\/blog_technical_images\/26249-insights_image-google-cloud-technical-blog-1-insights.jpg","field_partners":"","rxt_platforms":"Google Cloud Data","product":"","publisheddate":"2022-09-28T16:22:18","rxt_solutions":"Multicloud","opengraph_tag":"","x_tag":"","teaser_flag":"Technical Blog","seo_titile":"BigQuery APIs Reference | Rackspace Technology","field_topics":"","field_slug":"bigquery-apis-reference","footnotes":""},"categories":[178],"tags":[204],"class_list":["post-7083","blogs","type-blogs","status-publish","format-standard","has-post-thumbnail","hentry","category-blog","tag-technical"],"_links":{"self":[{"href":"https:\/\/www.rackspace.com\/de\/wp-json\/wp\/v2\/blogs\/7083","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.rackspace.com\/de\/wp-json\/wp\/v2\/blogs"}],"about":[{"href":"https:\/\/www.rackspace.com\/de\/wp-json\/wp\/v2\/types\/blogs"}],"author":[{"embeddable":true,"href":"https:\/\/www.rackspace.com\/de\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.rackspace.com\/de\/wp-json\/wp\/v2\/comments?post=7083"}],"version-history":[{"count":2,"href":"https:\/\/www.rackspace.com\/de\/wp-json\/wp\/v2\/blogs\/7083\/revisions"}],"predecessor-version":[{"id":8936,"href":"https:\/\/www.rackspace.com\/de\/wp-json\/wp\/v2\/blogs\/7083\/revisions\/8936"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.rackspace.com\/de\/wp-json\/wp\/v2\/media\/7085"}],"wp:attachment":[{"href":"https:\/\/www.rackspace.com\/de\/wp-json\/wp\/v2\/media?parent=7083"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.rackspace.com\/de\/wp-json\/wp\/v2\/categories?post=7083"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.rackspace.com\/de\/wp-json\/wp\/v2\/tags?post=7083"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}