Google Big Query view limitation on Python API
up vote
0
down vote
favorite
I am trying to create a view using the Python API for Google Big Query. I have the free trial and all the authentication setup. Whenever I create a view using the API only 25 rows are generated in the view, however when I create a view from the big query website all the rows are shown (3006). Is there a limit on rows when using API or might there be a problem in my code. This is the query that I am using (same was used in API and web):
query = "SELECT DISTINCT author, subreddit FROM `%s` WHERE subreddit = 'The_Donald'" %(TABLE+DATES[7])
google-cloud-platform google-bigquery google-python-api google-bigquery-ml
add a comment |
up vote
0
down vote
favorite
I am trying to create a view using the Python API for Google Big Query. I have the free trial and all the authentication setup. Whenever I create a view using the API only 25 rows are generated in the view, however when I create a view from the big query website all the rows are shown (3006). Is there a limit on rows when using API or might there be a problem in my code. This is the query that I am using (same was used in API and web):
query = "SELECT DISTINCT author, subreddit FROM `%s` WHERE subreddit = 'The_Donald'" %(TABLE+DATES[7])
google-cloud-platform google-bigquery google-python-api google-bigquery-ml
add a comment |
up vote
0
down vote
favorite
up vote
0
down vote
favorite
I am trying to create a view using the Python API for Google Big Query. I have the free trial and all the authentication setup. Whenever I create a view using the API only 25 rows are generated in the view, however when I create a view from the big query website all the rows are shown (3006). Is there a limit on rows when using API or might there be a problem in my code. This is the query that I am using (same was used in API and web):
query = "SELECT DISTINCT author, subreddit FROM `%s` WHERE subreddit = 'The_Donald'" %(TABLE+DATES[7])
google-cloud-platform google-bigquery google-python-api google-bigquery-ml
I am trying to create a view using the Python API for Google Big Query. I have the free trial and all the authentication setup. Whenever I create a view using the API only 25 rows are generated in the view, however when I create a view from the big query website all the rows are shown (3006). Is there a limit on rows when using API or might there be a problem in my code. This is the query that I am using (same was used in API and web):
query = "SELECT DISTINCT author, subreddit FROM `%s` WHERE subreddit = 'The_Donald'" %(TABLE+DATES[7])
google-cloud-platform google-bigquery google-python-api google-bigquery-ml
google-cloud-platform google-bigquery google-python-api google-bigquery-ml
edited Nov 8 at 10:32
asked Nov 8 at 10:24
hussamh10
558
558
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
up vote
0
down vote
I've just created a view with the Python Client Libraries using the following code:
from google.cloud import bigquery
client = bigquery.Client()
project = 'bigquery-samples'
source_dataset_id = 'reddit'
source_table_id = 'full'
shared_dataset_ref = client.dataset('my_dataset')
view_ref = shared_dataset_ref.table('my_shared_view')
view = bigquery.Table(view_ref)
sql_template = (
'SELECT DISTINCT author,subreddit_id FROM `{}.{}.{}` WHERE subreddit_id LIKE "%t5%"')
view.view_query = sql_template.format(
project, source_dataset_id, source_table_id)
view = client.create_table(view) # API request
print('Successfully created view at {}'.format(view.full_table_id))
And my view has 1359016 rows, verified by doing a:
SELECT COUNT(*) FROM `my_dataset.my_shared_view`
Hope it helps.
add a comment |
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
0
down vote
I've just created a view with the Python Client Libraries using the following code:
from google.cloud import bigquery
client = bigquery.Client()
project = 'bigquery-samples'
source_dataset_id = 'reddit'
source_table_id = 'full'
shared_dataset_ref = client.dataset('my_dataset')
view_ref = shared_dataset_ref.table('my_shared_view')
view = bigquery.Table(view_ref)
sql_template = (
'SELECT DISTINCT author,subreddit_id FROM `{}.{}.{}` WHERE subreddit_id LIKE "%t5%"')
view.view_query = sql_template.format(
project, source_dataset_id, source_table_id)
view = client.create_table(view) # API request
print('Successfully created view at {}'.format(view.full_table_id))
And my view has 1359016 rows, verified by doing a:
SELECT COUNT(*) FROM `my_dataset.my_shared_view`
Hope it helps.
add a comment |
up vote
0
down vote
I've just created a view with the Python Client Libraries using the following code:
from google.cloud import bigquery
client = bigquery.Client()
project = 'bigquery-samples'
source_dataset_id = 'reddit'
source_table_id = 'full'
shared_dataset_ref = client.dataset('my_dataset')
view_ref = shared_dataset_ref.table('my_shared_view')
view = bigquery.Table(view_ref)
sql_template = (
'SELECT DISTINCT author,subreddit_id FROM `{}.{}.{}` WHERE subreddit_id LIKE "%t5%"')
view.view_query = sql_template.format(
project, source_dataset_id, source_table_id)
view = client.create_table(view) # API request
print('Successfully created view at {}'.format(view.full_table_id))
And my view has 1359016 rows, verified by doing a:
SELECT COUNT(*) FROM `my_dataset.my_shared_view`
Hope it helps.
add a comment |
up vote
0
down vote
up vote
0
down vote
I've just created a view with the Python Client Libraries using the following code:
from google.cloud import bigquery
client = bigquery.Client()
project = 'bigquery-samples'
source_dataset_id = 'reddit'
source_table_id = 'full'
shared_dataset_ref = client.dataset('my_dataset')
view_ref = shared_dataset_ref.table('my_shared_view')
view = bigquery.Table(view_ref)
sql_template = (
'SELECT DISTINCT author,subreddit_id FROM `{}.{}.{}` WHERE subreddit_id LIKE "%t5%"')
view.view_query = sql_template.format(
project, source_dataset_id, source_table_id)
view = client.create_table(view) # API request
print('Successfully created view at {}'.format(view.full_table_id))
And my view has 1359016 rows, verified by doing a:
SELECT COUNT(*) FROM `my_dataset.my_shared_view`
Hope it helps.
I've just created a view with the Python Client Libraries using the following code:
from google.cloud import bigquery
client = bigquery.Client()
project = 'bigquery-samples'
source_dataset_id = 'reddit'
source_table_id = 'full'
shared_dataset_ref = client.dataset('my_dataset')
view_ref = shared_dataset_ref.table('my_shared_view')
view = bigquery.Table(view_ref)
sql_template = (
'SELECT DISTINCT author,subreddit_id FROM `{}.{}.{}` WHERE subreddit_id LIKE "%t5%"')
view.view_query = sql_template.format(
project, source_dataset_id, source_table_id)
view = client.create_table(view) # API request
print('Successfully created view at {}'.format(view.full_table_id))
And my view has 1359016 rows, verified by doing a:
SELECT COUNT(*) FROM `my_dataset.my_shared_view`
Hope it helps.
answered Nov 8 at 15:10
F10
1,1281414
1,1281414
add a comment |
add a comment |
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53205776%2fgoogle-big-query-view-limitation-on-python-api%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown