程序员子悠 · 好记性不如烂笔头

Command任务

1. 创建tpye=command的job文件

silence-pro:command silence$ ll
total 40
-rw-r--r--  1 silence  wheel  61  7  3 21:38 command1.job
-rw-r--r--  1 silence  wheel  74  7  6 17:56 command2.job
-rw-r--r--  1 silence  wheel  68  7  6 17:59 command3.job
-rw-r--r--  1 silence  wheel  67  7  6 17:57 command4.job
-rw-r--r--  1 silence  wheel  74  7  6 18:00 command5.job
silence-pro:command silence$ cat command1.job
type=command
command=echo "create directory before get data"
silence-pro:command silence$ cat command2.job
type=command
command=echo "get data from logserver"
dependencies=command1
silence-pro:command silence$ cat command3.job
type=command
command=echo "get data from ftp"
dependencies=command1
silence-pro:command silence$ cat command4.job
type=command
command=echo "insert into hive"
dependencies=command5
silence-pro:command silence$ cat command5.job
type=command
command=echo "upload to hdfs"
dependencies=command3,command2
silence-pro:command silence$

2. 将文件夹压缩为zip包

silence-pro:usercase silence$ ll
total 8
drwxr-xr-x  7 silence  wheel   224  7  6 18:00 command
drwxr-xr-x  4 silence  wheel   128  7  6 17:48 java
-rw-r--r--  1 silence  wheel  3021  7  6 17:48 java.zip
silence-pro:usercase silence$ zip -r command.zip command/
  adding: command/ (stored 0%)
  adding: command/command2.job (deflated 14%)
  adding: command/command3.job (deflated 15%)
  adding: command/command1.job (deflated 7%)
  adding: command/command4.job (deflated 15%)
  adding: command/command5.job (deflated 20%)
silence-pro:usercase silence$ ll
total 16
drwxr-xr-x  7 silence  wheel   224  7  6 18:00 command
-rw-r--r--  1 silence  wheel  1301  7  6 18:04 command.zip
drwxr-xr-x  4 silence  wheel   128  7  6 17:48 java
-rw-r--r--  1 silence  wheel  3021  7  6 17:48 java.zip
silence-pro:usercase silence$

3. 在Azkaban系统创建项目并上传压缩包
   
   
   
   

command: 需要执行的命令
dependencies: job的依赖关系

Java任务

1. 创建java.job

silence-pro:java silence$ cat java.job
type=javaprocess
#指定类的全路径
java.class=com.coocaa.ad.azkaban.demo.DemoAzkaban
#指定执行jar包的路径
classpath=lib/*
silence-pro:java silence$

2. 编写java文件

package com.coocaa.ad.azkaban.demo;
/**
 * <br>
 * <b>Function：</b><br>
 * <b>Author：</b>@author Silence<br>
 * <b>Date：</b>2018-07-04 16:54<br>
 * <b>Desc：</b>无<br>
 */
public class DemoAzkaban {

    public static void main(String[] args) {
        System.out.println("hello world...");
    }
}

3. 讲上述文件打包成jar放到java.job 同级的lib文件夹下

silence-pro:java silence$ ll
total 8
-rw-r--r--  1 silence  wheel  135  7  6 17:48 java.job
drwxr-xr-x  3 silence  wheel   96  7  6 17:46 lib
silence-pro:java silence$ cd lib/
silence-pro:lib silence$ ll
total 8
-rw-r--r--  1 silence  wheel  3634  7  6 17:46 adx-azkaban-0.0.1-SNAPSHOT.jar
silence-pro:lib silence$

4. 创建Java-project项目并压缩成zip包并上传系统
   
5. 执行任务
   

type=javaprocess Java任务类型 java-type
java.class=com.coocaa.ad.azkaban.demo.DemoAzkaban 配置类的全路径
classpath=lib/* 存放依赖包