2009-12-03T18:25:29+00:00 It appears that dumbo cat /hdfs/path/part* does not actually concatenate all of the parts in an HDFS directory -- instead, it silently emits only the key-value pairs from the first part. Since the normal Dumbo syntax without the final star chokes on the _logs directory that Hadoop creates by default, people may be using this part* syntax frequently, and they may not realize that it yields incorrect results. Current workarounds include using dumbo cat without the star by manually deleting the _logs directory or configuring Hadoop not to create it. It may be more convenient to use the HDFS ls command to iterate through the part files in a directory explicitly to ensure that each one is processed as expected. false 717835 0 false 66 2 ceBprigzar261JaaeP0Qfc cLQIPQdLOr3QNweJe5aVNr 0 0

dumbo cat /hdfs/path/part* silently fails to concatenate all part files

2009-12-03T18:25:29+00:00 bZobO4dLOr3Qu8eJe5aVNr 2009-03-11T16:49:12+00:00 Dumbo should provide a @-root /some/path@ option that makes all @-input another/path@ options relative to the specified root. Moreover, it could automatically add path-dependant options like @-inputformat@ when such a root is given, getting the <code>root->option</code> mappings from @/etc/dumbo.conf@ or @~/.dumborc@. false 302576 0 false 1 3 bZobO4dLOr3Qu8eJe5aVNr cLQIPQdLOr3QNweJe5aVNr 0 0

Roots and mappings

2009-03-11T16:49:12+00:00 3 2009-06-19T14:11:05+00:00 Dumbo has a streamoutput option that, if set, overrides stream.reduce.out (or stream.map.output if there is no reducer). It would be neat to be able to specify both the streamoutput option and stream.[map|reduce].output. false 410588 0 false 49 3 b7cRxogf0r3Os9eJe5afGb cLQIPQdLOr3QNweJe5aVNr 0 0

Add option to force read/write of typedbytes

2009-06-19T14:11:05+00:00 enhancement 2009-06-26T14:46:10+00:00 Since starters can parse command line options with delopt, it'd be useful to be able to specify a job on the command line and setup the appropriate runner, or have all jobs run sequentially, optionally with different output files. This would allow you to define within one file, several map/reduce methods in subclasses of some shared class that defines the shared functionality (e.g. parsing, reducer) false 418699 0 false 51 3 cpprOYyL4r3ONCeJe5aVNr cLQIPQdLOr3QNweJe5aVNr 0 0

Allow running multiple jobs from the same original input from one script

2009-06-26T14:46:10+00:00 enhancement bZobO4dLOr3Qu8eJe5aVNr 2009-09-15T11:02:30+00:00 When @dumbo start ./someprog.py ...@ is executed and the file @someprog.py@ doesn't exist, Dumbo will return the error "relative module names not supported" which isn't very helpful. false 517763 0 false 60 3 bZobO4dLOr3Qu8eJe5aVNr cLQIPQdLOr3QNweJe5aVNr 0 0

cryptic "relative module names not supported" error message

2009-09-15T11:02:30+00:00 2009-10-01T11:12:49+00:00 The following scripts demonstrate a failure to fail when executed on a hadoop cluster (fails fine if executed locally). The test uses dumbo.sumsreducer where dumbo.sumreduce should be used. A TypeError should be thrown by dumbo.sumsreducer (in the combiner). Instead the Hadoop reports show no error and zero output from the mapper. Run the test: "python run.py" false 539934 0 false 61 3 d1oDK2eIKr3OVqeJe5aVNr cLQIPQdLOr3QNweJe5aVNr 0 0

Failure to fail

2009-10-01T11:12:49+00:00 bZobO4dLOr3Qu8eJe5aVNr 2010-01-13T16:46:27+00:00 It would be useful if a multimapper would inherit all the opt decorations of the mappers that get added to it. false 832499 0 false 68 3 bZobO4dLOr3Qu8eJe5aVNr cLQIPQdLOr3QNweJe5aVNr 0 0

make MultiMapper inherit options

2010-01-13T16:46:27+00:00