Python Multiprocessing( TypeError: cannot serialize '_io.BufferedReader' object )











up vote
1
down vote

favorite












I'm trying to make dictionary attack on zip file using Pool to increase speed.
But I face next error in Python 3.6, while it works in Python 2.7:



Traceback (most recent call last):
File "zip_crack.py", line 42, in <module>
main()
File "zip_crack.py", line 28, in main
for result in results:
File "/usr/lib/python3.6/multiprocessing/pool.py", line 761, in next
raise value
File "/usr/lib/python3.6/multiprocessing/pool.py", line 450, in _ handle_tasks
put(task)
File "/usr/lib/python3.6/multiprocessing/connection.py", line 206, in send
self._send_bytes(_ForkingPickler.dumps(obj))
File "/usr/lib/python3.6/multiprocessing/reduction.py", line 51, in dumps
cls(buf, protocol).dump(obj)
TypeError: cannot serialize '_io.BufferedReader' object


I tried to search for same errors but couldn't find answer that can help here.



Code looks like this



def crack(pwd, f):
try:
key = pwd.strip()
f.extractall(pwd=key)
return True
except:
pass

z_file = zipfile.ZipFile("../folder.zip")

with open('words.dic', 'r') as passes:
start = time.time()

lines = passes.readlines()

pool = Pool(50)

results = pool.imap_unordered(partial(crack, f=z_file), lines)
pool.close()

for result in results:
if result:
pool.terminate()
break
pool.join()


I also tried another approach using map



with contextlib.closing(Pool(50)) as pool:

pool.map(partial(crack, f=z_file), lines)


which worked great and found passwords quickly in Python 2.7 but it throws same exception in python 3.6










share|improve this question




























    up vote
    1
    down vote

    favorite












    I'm trying to make dictionary attack on zip file using Pool to increase speed.
    But I face next error in Python 3.6, while it works in Python 2.7:



    Traceback (most recent call last):
    File "zip_crack.py", line 42, in <module>
    main()
    File "zip_crack.py", line 28, in main
    for result in results:
    File "/usr/lib/python3.6/multiprocessing/pool.py", line 761, in next
    raise value
    File "/usr/lib/python3.6/multiprocessing/pool.py", line 450, in _ handle_tasks
    put(task)
    File "/usr/lib/python3.6/multiprocessing/connection.py", line 206, in send
    self._send_bytes(_ForkingPickler.dumps(obj))
    File "/usr/lib/python3.6/multiprocessing/reduction.py", line 51, in dumps
    cls(buf, protocol).dump(obj)
    TypeError: cannot serialize '_io.BufferedReader' object


    I tried to search for same errors but couldn't find answer that can help here.



    Code looks like this



    def crack(pwd, f):
    try:
    key = pwd.strip()
    f.extractall(pwd=key)
    return True
    except:
    pass

    z_file = zipfile.ZipFile("../folder.zip")

    with open('words.dic', 'r') as passes:
    start = time.time()

    lines = passes.readlines()

    pool = Pool(50)

    results = pool.imap_unordered(partial(crack, f=z_file), lines)
    pool.close()

    for result in results:
    if result:
    pool.terminate()
    break
    pool.join()


    I also tried another approach using map



    with contextlib.closing(Pool(50)) as pool:

    pool.map(partial(crack, f=z_file), lines)


    which worked great and found passwords quickly in Python 2.7 but it throws same exception in python 3.6










    share|improve this question


























      up vote
      1
      down vote

      favorite









      up vote
      1
      down vote

      favorite











      I'm trying to make dictionary attack on zip file using Pool to increase speed.
      But I face next error in Python 3.6, while it works in Python 2.7:



      Traceback (most recent call last):
      File "zip_crack.py", line 42, in <module>
      main()
      File "zip_crack.py", line 28, in main
      for result in results:
      File "/usr/lib/python3.6/multiprocessing/pool.py", line 761, in next
      raise value
      File "/usr/lib/python3.6/multiprocessing/pool.py", line 450, in _ handle_tasks
      put(task)
      File "/usr/lib/python3.6/multiprocessing/connection.py", line 206, in send
      self._send_bytes(_ForkingPickler.dumps(obj))
      File "/usr/lib/python3.6/multiprocessing/reduction.py", line 51, in dumps
      cls(buf, protocol).dump(obj)
      TypeError: cannot serialize '_io.BufferedReader' object


      I tried to search for same errors but couldn't find answer that can help here.



      Code looks like this



      def crack(pwd, f):
      try:
      key = pwd.strip()
      f.extractall(pwd=key)
      return True
      except:
      pass

      z_file = zipfile.ZipFile("../folder.zip")

      with open('words.dic', 'r') as passes:
      start = time.time()

      lines = passes.readlines()

      pool = Pool(50)

      results = pool.imap_unordered(partial(crack, f=z_file), lines)
      pool.close()

      for result in results:
      if result:
      pool.terminate()
      break
      pool.join()


      I also tried another approach using map



      with contextlib.closing(Pool(50)) as pool:

      pool.map(partial(crack, f=z_file), lines)


      which worked great and found passwords quickly in Python 2.7 but it throws same exception in python 3.6










      share|improve this question















      I'm trying to make dictionary attack on zip file using Pool to increase speed.
      But I face next error in Python 3.6, while it works in Python 2.7:



      Traceback (most recent call last):
      File "zip_crack.py", line 42, in <module>
      main()
      File "zip_crack.py", line 28, in main
      for result in results:
      File "/usr/lib/python3.6/multiprocessing/pool.py", line 761, in next
      raise value
      File "/usr/lib/python3.6/multiprocessing/pool.py", line 450, in _ handle_tasks
      put(task)
      File "/usr/lib/python3.6/multiprocessing/connection.py", line 206, in send
      self._send_bytes(_ForkingPickler.dumps(obj))
      File "/usr/lib/python3.6/multiprocessing/reduction.py", line 51, in dumps
      cls(buf, protocol).dump(obj)
      TypeError: cannot serialize '_io.BufferedReader' object


      I tried to search for same errors but couldn't find answer that can help here.



      Code looks like this



      def crack(pwd, f):
      try:
      key = pwd.strip()
      f.extractall(pwd=key)
      return True
      except:
      pass

      z_file = zipfile.ZipFile("../folder.zip")

      with open('words.dic', 'r') as passes:
      start = time.time()

      lines = passes.readlines()

      pool = Pool(50)

      results = pool.imap_unordered(partial(crack, f=z_file), lines)
      pool.close()

      for result in results:
      if result:
      pool.terminate()
      break
      pool.join()


      I also tried another approach using map



      with contextlib.closing(Pool(50)) as pool:

      pool.map(partial(crack, f=z_file), lines)


      which worked great and found passwords quickly in Python 2.7 but it throws same exception in python 3.6







      python-3.x multiprocessing pool






      share|improve this question















      share|improve this question













      share|improve this question




      share|improve this question








      edited Nov 9 at 21:06

























      asked Nov 9 at 20:52









      V. Khud

      134




      134





























          active

          oldest

          votes











          Your Answer






          StackExchange.ifUsing("editor", function () {
          StackExchange.using("externalEditor", function () {
          StackExchange.using("snippets", function () {
          StackExchange.snippets.init();
          });
          });
          }, "code-snippets");

          StackExchange.ready(function() {
          var channelOptions = {
          tags: "".split(" "),
          id: "1"
          };
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function() {
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled) {
          StackExchange.using("snippets", function() {
          createEditor();
          });
          }
          else {
          createEditor();
          }
          });

          function createEditor() {
          StackExchange.prepareEditor({
          heartbeatType: 'answer',
          convertImagesToLinks: true,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: 10,
          bindNavPrevention: true,
          postfix: "",
          imageUploader: {
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          },
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          });


          }
          });














          draft saved

          draft discarded


















          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53233113%2fpython-multiprocessing-typeerror-cannot-serialize-io-bufferedreader-object%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown






























          active

          oldest

          votes













          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes
















          draft saved

          draft discarded




















































          Thanks for contributing an answer to Stack Overflow!


          • Please be sure to answer the question. Provide details and share your research!

          But avoid



          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.


          To learn more, see our tips on writing great answers.





          Some of your past answers have not been well-received, and you're in danger of being blocked from answering.


          Please pay close attention to the following guidance:


          • Please be sure to answer the question. Provide details and share your research!

          But avoid



          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.


          To learn more, see our tips on writing great answers.




          draft saved


          draft discarded














          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53233113%2fpython-multiprocessing-typeerror-cannot-serialize-io-bufferedreader-object%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown