r*****d
发帖数: 346 | 1 请问大家：有什么好方法好资料帮助学习Pig Latin? Pig Latin我完全是新手。我在
amazon上找了下，好像没有书评特别好的。
谢谢！ | c***z
发帖数: 6348 | 2 You need the following things:
1. An editor, I use sublime2, the cloudera package uses Gedit
2. A cluster with Pig installed at the edge nodes, you can use the VM in the
cloudera package
3. A file transfer to move Pig code from local drive (if you edit locally)
to edge node, I use Winscp, the cloudera package uses Hue
4. A way to run the code at edge node, I use putty, the cloudera package
uses Hue
My work flow: write Pig code locally using sublime2, upload code to edge
node using winscp, run code at edge node using putty. | c***z
发帖数: 6348 | 3 Also, my work flow using Scala Scalding (i.e. Scala on Hadoop):
1. edit and compile in Intelij, or other IDE, or edit in any text editor and
compile in a terminal (CMD for windows)
setting up Intelij is complicated and out of my scope
2. upload the jar file to edge node, I use winscp
optionally, upload the code to github for version control
3. run the jar file at edge node using putty, with specific input path,
output path and other argument, I save them to an .sh file for reuse (you
can save to .txt file and then copy and paste)
the command to run the jar is something like
hadoop jar myjar.jar packagename.functionname --input "myinputpath/part*" --
output "myoutputpath" --hdfs
4. to get the output to a text file, use something like
hadoop fs -cat "myoutputpath/part*" > myresult.tsv
(I prefer .tsv over .csv because comma can appear in numbers like 133,010
and mess up things) | g*******t
发帖数: 7704 | | s*********e
发帖数: 1051 | 5 对大数据处理，pig不慢。你自己测测时间，与r或python比一下
【在 g*******t 的大作中提到】
: pig好慢好慢
| l*******m
发帖数: 1096 | 6 online tutorials are good enough for start if you know SQL and a bit
mapreduce
【在 r*****d 的大作中提到】
: 请问大家：有什么好方法好资料帮助学习Pig Latin? Pig Latin我完全是新手。我在
: amazon上找了下，好像没有书评特别好的。
: 谢谢！
| r*****d
发帖数: 346 | 7 谢谢大家回复！
update: i found the book 'Programming Pig' especially the first 6 chapters
very helpful - it is not demanding to pick up Pig as yet another tool :) |
|
