Google Big Query view limitation on Python API











up vote
0
down vote

favorite












I am trying to create a view using the Python API for Google Big Query. I have the free trial and all the authentication setup. Whenever I create a view using the API only 25 rows are generated in the view, however when I create a view from the big query website all the rows are shown (3006). Is there a limit on rows when using API or might there be a problem in my code. This is the query that I am using (same was used in API and web):



query = "SELECT DISTINCT author, subreddit FROM `%s` WHERE subreddit = 'The_Donald'" %(TABLE+DATES[7])









share|improve this question




























    up vote
    0
    down vote

    favorite












    I am trying to create a view using the Python API for Google Big Query. I have the free trial and all the authentication setup. Whenever I create a view using the API only 25 rows are generated in the view, however when I create a view from the big query website all the rows are shown (3006). Is there a limit on rows when using API or might there be a problem in my code. This is the query that I am using (same was used in API and web):



    query = "SELECT DISTINCT author, subreddit FROM `%s` WHERE subreddit = 'The_Donald'" %(TABLE+DATES[7])









    share|improve this question


























      up vote
      0
      down vote

      favorite









      up vote
      0
      down vote

      favorite











      I am trying to create a view using the Python API for Google Big Query. I have the free trial and all the authentication setup. Whenever I create a view using the API only 25 rows are generated in the view, however when I create a view from the big query website all the rows are shown (3006). Is there a limit on rows when using API or might there be a problem in my code. This is the query that I am using (same was used in API and web):



      query = "SELECT DISTINCT author, subreddit FROM `%s` WHERE subreddit = 'The_Donald'" %(TABLE+DATES[7])









      share|improve this question















      I am trying to create a view using the Python API for Google Big Query. I have the free trial and all the authentication setup. Whenever I create a view using the API only 25 rows are generated in the view, however when I create a view from the big query website all the rows are shown (3006). Is there a limit on rows when using API or might there be a problem in my code. This is the query that I am using (same was used in API and web):



      query = "SELECT DISTINCT author, subreddit FROM `%s` WHERE subreddit = 'The_Donald'" %(TABLE+DATES[7])






      google-cloud-platform google-bigquery google-python-api google-bigquery-ml






      share|improve this question















      share|improve this question













      share|improve this question




      share|improve this question








      edited Nov 8 at 10:32

























      asked Nov 8 at 10:24









      hussamh10

      558




      558
























          1 Answer
          1






          active

          oldest

          votes

















          up vote
          0
          down vote













          I've just created a view with the Python Client Libraries using the following code:



          from google.cloud import bigquery
          client = bigquery.Client()
          project = 'bigquery-samples'
          source_dataset_id = 'reddit'
          source_table_id = 'full'
          shared_dataset_ref = client.dataset('my_dataset')
          view_ref = shared_dataset_ref.table('my_shared_view')
          view = bigquery.Table(view_ref)
          sql_template = (
          'SELECT DISTINCT author,subreddit_id FROM `{}.{}.{}` WHERE subreddit_id LIKE "%t5%"')
          view.view_query = sql_template.format(
          project, source_dataset_id, source_table_id)
          view = client.create_table(view) # API request

          print('Successfully created view at {}'.format(view.full_table_id))


          And my view has 1359016 rows, verified by doing a:



          SELECT COUNT(*) FROM `my_dataset.my_shared_view` 


          Hope it helps.






          share|improve this answer





















            Your Answer






            StackExchange.ifUsing("editor", function () {
            StackExchange.using("externalEditor", function () {
            StackExchange.using("snippets", function () {
            StackExchange.snippets.init();
            });
            });
            }, "code-snippets");

            StackExchange.ready(function() {
            var channelOptions = {
            tags: "".split(" "),
            id: "1"
            };
            initTagRenderer("".split(" "), "".split(" "), channelOptions);

            StackExchange.using("externalEditor", function() {
            // Have to fire editor after snippets, if snippets enabled
            if (StackExchange.settings.snippets.snippetsEnabled) {
            StackExchange.using("snippets", function() {
            createEditor();
            });
            }
            else {
            createEditor();
            }
            });

            function createEditor() {
            StackExchange.prepareEditor({
            heartbeatType: 'answer',
            convertImagesToLinks: true,
            noModals: true,
            showLowRepImageUploadWarning: true,
            reputationToPostImages: 10,
            bindNavPrevention: true,
            postfix: "",
            imageUploader: {
            brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
            contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
            allowUrls: true
            },
            onDemand: true,
            discardSelector: ".discard-answer"
            ,immediatelyShowMarkdownHelp:true
            });


            }
            });














             

            draft saved


            draft discarded


















            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53205776%2fgoogle-big-query-view-limitation-on-python-api%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown

























            1 Answer
            1






            active

            oldest

            votes








            1 Answer
            1






            active

            oldest

            votes









            active

            oldest

            votes






            active

            oldest

            votes








            up vote
            0
            down vote













            I've just created a view with the Python Client Libraries using the following code:



            from google.cloud import bigquery
            client = bigquery.Client()
            project = 'bigquery-samples'
            source_dataset_id = 'reddit'
            source_table_id = 'full'
            shared_dataset_ref = client.dataset('my_dataset')
            view_ref = shared_dataset_ref.table('my_shared_view')
            view = bigquery.Table(view_ref)
            sql_template = (
            'SELECT DISTINCT author,subreddit_id FROM `{}.{}.{}` WHERE subreddit_id LIKE "%t5%"')
            view.view_query = sql_template.format(
            project, source_dataset_id, source_table_id)
            view = client.create_table(view) # API request

            print('Successfully created view at {}'.format(view.full_table_id))


            And my view has 1359016 rows, verified by doing a:



            SELECT COUNT(*) FROM `my_dataset.my_shared_view` 


            Hope it helps.






            share|improve this answer

























              up vote
              0
              down vote













              I've just created a view with the Python Client Libraries using the following code:



              from google.cloud import bigquery
              client = bigquery.Client()
              project = 'bigquery-samples'
              source_dataset_id = 'reddit'
              source_table_id = 'full'
              shared_dataset_ref = client.dataset('my_dataset')
              view_ref = shared_dataset_ref.table('my_shared_view')
              view = bigquery.Table(view_ref)
              sql_template = (
              'SELECT DISTINCT author,subreddit_id FROM `{}.{}.{}` WHERE subreddit_id LIKE "%t5%"')
              view.view_query = sql_template.format(
              project, source_dataset_id, source_table_id)
              view = client.create_table(view) # API request

              print('Successfully created view at {}'.format(view.full_table_id))


              And my view has 1359016 rows, verified by doing a:



              SELECT COUNT(*) FROM `my_dataset.my_shared_view` 


              Hope it helps.






              share|improve this answer























                up vote
                0
                down vote










                up vote
                0
                down vote









                I've just created a view with the Python Client Libraries using the following code:



                from google.cloud import bigquery
                client = bigquery.Client()
                project = 'bigquery-samples'
                source_dataset_id = 'reddit'
                source_table_id = 'full'
                shared_dataset_ref = client.dataset('my_dataset')
                view_ref = shared_dataset_ref.table('my_shared_view')
                view = bigquery.Table(view_ref)
                sql_template = (
                'SELECT DISTINCT author,subreddit_id FROM `{}.{}.{}` WHERE subreddit_id LIKE "%t5%"')
                view.view_query = sql_template.format(
                project, source_dataset_id, source_table_id)
                view = client.create_table(view) # API request

                print('Successfully created view at {}'.format(view.full_table_id))


                And my view has 1359016 rows, verified by doing a:



                SELECT COUNT(*) FROM `my_dataset.my_shared_view` 


                Hope it helps.






                share|improve this answer












                I've just created a view with the Python Client Libraries using the following code:



                from google.cloud import bigquery
                client = bigquery.Client()
                project = 'bigquery-samples'
                source_dataset_id = 'reddit'
                source_table_id = 'full'
                shared_dataset_ref = client.dataset('my_dataset')
                view_ref = shared_dataset_ref.table('my_shared_view')
                view = bigquery.Table(view_ref)
                sql_template = (
                'SELECT DISTINCT author,subreddit_id FROM `{}.{}.{}` WHERE subreddit_id LIKE "%t5%"')
                view.view_query = sql_template.format(
                project, source_dataset_id, source_table_id)
                view = client.create_table(view) # API request

                print('Successfully created view at {}'.format(view.full_table_id))


                And my view has 1359016 rows, verified by doing a:



                SELECT COUNT(*) FROM `my_dataset.my_shared_view` 


                Hope it helps.







                share|improve this answer












                share|improve this answer



                share|improve this answer










                answered Nov 8 at 15:10









                F10

                1,1281414




                1,1281414






























                     

                    draft saved


                    draft discarded



















































                     


                    draft saved


                    draft discarded














                    StackExchange.ready(
                    function () {
                    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53205776%2fgoogle-big-query-view-limitation-on-python-api%23new-answer', 'question_page');
                    }
                    );

                    Post as a guest















                    Required, but never shown





















































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown

































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown







                    Popular posts from this blog

                    Schultheiß

                    Liste der Kulturdenkmale in Wilsdruff

                    Android Play Services Check