Hadoop Mapreduce Flow Custom InputFormat, RecordReader [closed]











up vote
1
down vote

favorite












I'm new to Hadoop and currently I'm learning mapreduce design pattern from Donald Miner & Adam Shook MapReduce Design Pattern book. So in this book there is Cartesian Product Join Pattern and it create a custom InputFormat and RecordReader and it makes me confuse about the flow. I have many question about the code :




  1. Where is InputFormat and RecordReader code actually run? Is it in AppMaster for InputFormat and NodeManager for RecordReader?

  2. Why is there another InputFormat instance in the CartesianRecordReader? In all MapReduce flow that I found, InputFormat is running before record reader does.


Here is the source code
https://github.com/adamjshook/mapreducepatterns/blob/master/MRDP/src/main/java/mrdp/ch5/CartesianProduct.java



That's all, thanks in advance :)






java hadoop design-patterns mapreduce







share|improve this question















closed as too broad by cricket_007, GBlodgett, Graham, Shiladitya, Matthew L Daniel Nov 11 at 5:00


Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.



















    up vote
    1
    down vote

    favorite












    I'm new to Hadoop and currently I'm learning mapreduce design pattern from Donald Miner & Adam Shook MapReduce Design Pattern book. So in this book there is Cartesian Product Join Pattern and it create a custom InputFormat and RecordReader and it makes me confuse about the flow. I have many question about the code :




    1. Where is InputFormat and RecordReader code actually run? Is it in AppMaster for InputFormat and NodeManager for RecordReader?

    2. Why is there another InputFormat instance in the CartesianRecordReader? In all MapReduce flow that I found, InputFormat is running before record reader does.


    Here is the source code
    https://github.com/adamjshook/mapreducepatterns/blob/master/MRDP/src/main/java/mrdp/ch5/CartesianProduct.java



    That's all, thanks in advance :)






    java hadoop design-patterns mapreduce







    share|improve this question















    closed as too broad by cricket_007, GBlodgett, Graham, Shiladitya, Matthew L Daniel Nov 11 at 5:00


    Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

















      up vote
      1
      down vote

      favorite









      up vote
      1
      down vote

      favorite











      I'm new to Hadoop and currently I'm learning mapreduce design pattern from Donald Miner & Adam Shook MapReduce Design Pattern book. So in this book there is Cartesian Product Join Pattern and it create a custom InputFormat and RecordReader and it makes me confuse about the flow. I have many question about the code :




      1. Where is InputFormat and RecordReader code actually run? Is it in AppMaster for InputFormat and NodeManager for RecordReader?

      2. Why is there another InputFormat instance in the CartesianRecordReader? In all MapReduce flow that I found, InputFormat is running before record reader does.


      Here is the source code
      https://github.com/adamjshook/mapreducepatterns/blob/master/MRDP/src/main/java/mrdp/ch5/CartesianProduct.java



      That's all, thanks in advance :)






      java hadoop design-patterns mapreduce







      share|improve this question















      I'm new to Hadoop and currently I'm learning mapreduce design pattern from Donald Miner & Adam Shook MapReduce Design Pattern book. So in this book there is Cartesian Product Join Pattern and it create a custom InputFormat and RecordReader and it makes me confuse about the flow. I have many question about the code :




      1. Where is InputFormat and RecordReader code actually run? Is it in AppMaster for InputFormat and NodeManager for RecordReader?

      2. Why is there another InputFormat instance in the CartesianRecordReader? In all MapReduce flow that I found, InputFormat is running before record reader does.


      Here is the source code
      https://github.com/adamjshook/mapreducepatterns/blob/master/MRDP/src/main/java/mrdp/ch5/CartesianProduct.java



      That's all, thanks in advance :)






      java hadoop design-patterns mapreduce




      java hadoop design-patterns mapreduce






      share|improve this question















      share|improve this question













      share|improve this question




      share|improve this question








      edited Nov 13 at 16:14

























      asked Nov 9 at 16:07









      Irvan

      3617




      3617




      closed as too broad by cricket_007, GBlodgett, Graham, Shiladitya, Matthew L Daniel Nov 11 at 5:00


      Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.






      closed as too broad by cricket_007, GBlodgett, Graham, Shiladitya, Matthew L Daniel Nov 11 at 5:00


      Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.































          active

          oldest

          votes






















          active

          oldest

          votes













          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes

          -

          Popular posts from this blog

          Schultheiß

          Image
          Der Titel dieses Artikels ist mehrdeutig. Weitere Bedeutungen sind unter Schultheiß (Begriffsklärung) aufgeführt. Ein Schultheiß. Holzschnitt von Peter Flötner (16. Jahrhundert) Der Schultheiß oder Schuldheiß (von althochdeutsch  sculdheizo ‚Leistung Befehlender‘, vgl. mittelniederdeutsch  schult(h)ēte , latinisiert (mittellat.) sculte(t)us , schwäbisch heute noch Schultes für „Bürgermeister“) bezeichnet einen in vielen westgermanischen Rechtsordnungen vorgesehenen Beamten, der Schuld heischt : Er hatte im Auftrag seines Herren (Landesherrn, Stadtherrn, Grundherrn) die Mitglieder einer Gemeinde zur Leistung ihrer Schuldigkeit anzuhalten, also Abgaben einzuziehen oder für das Beachten anderer Verpflichtungen Sorge zu tragen. Sprachliche Varianten des Schultheißen sind Schulte , Schultes oder Schulze . Früher wurde zwischen dem Stadtschulzen und dem Dorfschulzen unterschieden. In der städtischen Gerichts- und Gemeindeverfassung war er ein vom städtischen Rat oder vom L
          Read more

          Liste der Kulturdenkmale in Wilsdruff

          Image
          Wappen von Wilsdruff Die Liste der Kulturdenkmale in Wilsdruff enthält die Kulturdenkmale in Wilsdruff. Die Anmerkungen sind zu beachten. Diese Liste ist eine Teilliste der Liste der Kulturdenkmale im Landkreis Sächsische Schweiz-Osterzgebirge. Diese Liste ist eine Teilliste der Liste der Kulturdenkmale in Sachsen. Inhaltsverzeichnis 1 Legende 2 Wilsdruff 3 Birkenhain 4 Blankenstein 5 Braunsdorf 6 Grumbach 7 Grund 8 Helbigsdorf 9 Herzogswalde 10 Kaufbach 11 Kesselsdorf 12 Kleinopitz 13 Limbach 14 Mohorn 15 Oberhermsdorf 16 Anmerkungen 17 Ausführliche Denkmaltexte 18 Quellen 19 Weblinks Legende | Bild: zeigt ein Bild des Kulturdenkmals und gegebenenfalls einen Link zu weiteren Fotos des Kulturdenkmals Bezeichnung: Name, Bezeichnung oder die Art des Kulturdenkmals Lage: Straßenname und wenn vorhanden Hausnummer des Kulturdenkmals; Grundsortierung der Liste erfolgt nach dieser A
          Read more

          Android Play Services Check

          Image
          up vote 0 down vote favorite Im facing a problem currently im making my app to run on t.v its working fine on mobiles. but the issue is if i run my app on t.v emulator splash screen appears then the app crashes log shows this error. No acceptable module found. Local version is 0 and remote version is 0 i googled this error and came with a solution that my emulator should have latest google play services installed in it.. but in my t.v emulator there is no play store so i cant update my play services to latest version. so is there any way to make a check that if play services version is less then this so run the app instead of crashing, and if the play services version is latest also run the aap... here is my code where im getting the play services version.. int playServicesAvailable = GoogleApiAvailability.g
          Read more