使用rabbitmq时,保证消息的可靠性投递【四】

上期已经配置和运行了esjob,那么可靠性投递已经接近完成,为了将配置和运行做的更加自动化,对此进行一个升级。

  1. 先看依赖pom文件
<properties>
        <elastic-job.version>2.1.4</elastic-job.version>
    </properties>
    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter</artifactId>
        </dependency>
        <!--自动装配需要-->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-configuration-processor</artifactId>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>com.dangdang</groupId>
            <artifactId>elastic-job-lite-core</artifactId>
            <version>${elastic-job.version}</version>
        </dependency>
        <dependency>
            <groupId>com.dangdang</groupId>
            <artifactId>elastic-job-lite-spring</artifactId>
            <version>${elastic-job.version}</version>
        </dependency>
    </dependencies>
  1. 创建自动装配类,在其中完成两个bean的创建:1. zookeeperReigister 2. 任务和任务调度。
    2-1. 由于是自动装配,所以在\resources\META-INF\spring.factories 中要配置

    org.springframework.boot.autoconfigure.EnableAutoConfiguration=\
    YOUR_PACKAGE.JobParserAutoConfiguration

    2-2. 贴代码

import com.dangdang.ddframe.job.reg.zookeeper.ZookeeperConfiguration;
import com.dangdang.ddframe.job.reg.zookeeper.ZookeeperRegistryCenter;
import com.zk.rabbit.task.paser.ElasticJobParser;
import lombok.extern.slf4j.Slf4j;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
// 此处为触发条件 在配置文件中 开头为 elastic.job.zk 并且包括两个 key {"namespace", "serverLists" 才会执行(serverLists 就是 zookeepeer的地址+端口号),否则matchIfMissing = false不执行
@ConditionalOnProperty(prefix = "elastic.job.zk", name = {"namespace", "serverLists"}, matchIfMissing = false)
// 读取配置类
@EnableConfigurationProperties(JobZookeeperProperties.class)
@Slf4j
public class JobParserAutoConfiguration {

    @Bean(initMethod = "init")
    public ZookeeperRegistryCenter zookeeperRegistryCenter(JobZookeeperProperties jobZookeeperProperties) {
        ZookeeperConfiguration zookeeperConfiguration = new ZookeeperConfiguration(jobZookeeperProperties.getServerLists(), jobZookeeperProperties.getNamespace());
        zookeeperConfiguration.setMaxRetries(jobZookeeperProperties.getMaxRetries());
        zookeeperConfiguration.setSessionTimeoutMilliseconds(jobZookeeperProperties.getSessionTimeoutMilliseconds());
        zookeeperConfiguration.setConnectionTimeoutMilliseconds(jobZookeeperProperties.getConnectionTimeoutMilliseconds());
        zookeeperConfiguration.setBaseSleepTimeMilliseconds(jobZookeeperProperties.getBaseSleepTimeMilliseconds());
        zookeeperConfiguration.setMaxSleepTimeMilliseconds(jobZookeeperProperties.getMaxSleepTimeMilliseconds());
        zookeeperConfiguration.setDigest(jobZookeeperProperties.getDigest());
        log.info("初始化job注册中心配置成功, zkaddress : {}, namespace : {}", jobZookeeperProperties.getServerLists(), jobZookeeperProperties.getNamespace());
        return new ZookeeperRegistryCenter(zookeeperConfiguration);
    }

    // es-job解析器
    @Bean
    public ElasticJobParser elasticJobParser(JobZookeeperProperties jobZookeeperProperties, ZookeeperRegistryCenter zookeeperRegistryCenter) {
        return new ElasticJobParser(jobZookeeperProperties, zookeeperRegistryCenter);
    }

}

2-3 我们先看看配置类 JobZookeeperProperties.class , 这里实际上把properties文件中的属性值读入到类对象中

import lombok.Data;
import org.springframework.boot.context.properties.ConfigurationProperties;

@ConfigurationProperties(prefix = "elastic.job.zk")
@Data
public class JobZookeeperProperties {
    private String namespace;

    private String serverLists;

    private int maxRetries = 3;

    private int connectionTimeoutMilliseconds = 15000;

    private int sessionTimeoutMilliseconds = 60000;

    private int baseSleepTimeMilliseconds = 1000;

    private int maxSleepTimeMilliseconds = 3000;

    private String digest = "";

}

2-4 再看esjob解析器都做了什么

2-4-1 解析器实现了ApplicationListener<ApplicationReadyEvent> 接口,即在整个spring容器加载好之后在运行,这样可以安全的获取容器已经搜集好的bean和beanDefinition。

public class ElasticJobParser implements ApplicationListener<ApplicationReadyEvent>{}

2-4-2 在构造方法中传入注册中心对象和配置对象

private JobZookeeperProperties jobZookeeperProperties;
    private ZookeeperRegistryCenter zookeeperRegistryCenter;

    public ElasticJobParser(JobZookeeperProperties jobZookeeperProperties, ZookeeperRegistryCenter zookeeperRegistryCenter) {
        this.jobZookeeperProperties = jobZookeeperProperties;
        this.zookeeperRegistryCenter = zookeeperRegistryCenter;
    }

2-4-3 接下来在ApplicationListener<ApplicationReadyEvent> 覆写方法 public void onApplicationEvent(ApplicationReadyEvent applicationReadyEvent)

3-4-5 在我们的设想中,自动注入JobParserAutoConfiguration一旦生效,方法onApplicationEvent 就开始执行,执行的目的即是创建各种esjob任务,上一期中每创建一个新的任务,就要写一遍配置类,现在省略这个过程,将配置过程就放在onApplicationEvent 中执行,相当于一个模板代码,而需要设置的属性,由注解传入。

3-4-6 先看看注解

@Target(ElementType.TYPE)
@Retention(RetentionPolicy.RUNTIME)
public @interface ElasticJobConfig {

    String name();    //elasticjob的名称

    String cron() default "";

    int shardingTotalCount() default 1;

    String shardingItemParameters() default "";

    String jobParameter() default "";

    boolean failover() default false;

    boolean misfire() default true;

    String description() default "";

    boolean overwrite() default false;

    boolean streamingProcess() default false;

    String scriptCommandLine() default "";

    boolean monitorExecution() default false;

    public int monitorPort() default -1;    //must

    public int maxTimeDiffSeconds() default -1;    //must

    public String jobShardingStrategyClass() default "";    //must

    public int reconcileIntervalMinutes() default 10;    //must

    public String eventTraceRdbDataSource() default "";    //must

    public String listener() default "";    //must

    public boolean disabled() default false;    //must

    public String distributedListener() default "";

    public long startedTimeoutMilliseconds() default Long.MAX_VALUE;    //must

    public long completedTimeoutMilliseconds() default Long.MAX_VALUE;        //must

    public String jobExceptionHandler() default "com.dangdang.ddframe.job.executor.handler.impl.DefaultJobExceptionHandler";

    public String executorServiceHandler() default "com.dangdang.ddframe.job.executor.handler.impl.DefaultExecutorServiceHandler";

}

都是任务的属性,前两个是必填。

3-6-7 回到onApplicationEvent中,将被 @ElasticJobConfig 注解过的类查出来遍历,注解类似

@ElasticJobConfig(
        name = "com.zk.rabbit.producer.task.RetryMessageDataflowJob",
        cron = "0/10 * * * * ?",
        description = "可靠性补偿消息任务",
        overwrite = true,
        shardingTotalCount = 1
)

3-6-8 遍历beans,并从其实现的接口推断出jobTypeName(DataflowJob,SimpleJob)

    ConfigurableApplicationContext context = applicationReadyEvent.getApplicationContext();
            Map<String, Object> beans = context.getBeansWithAnnotation(ElasticJobConfig.class);
            for (Iterator it = beans.values().iterator();it.hasNext();) {
                Object confBean = it.next();
                Class<?> clazz = confBean.getClass();
                if (clazz.getName().indexOf("$") > 0) {
                    String className = clazz.getName();
                    clazz = Class.forName(className.substring(0, className.indexOf("$")));
                }
                String jobTypeName = clazz.getInterfaces()[0].getSimpleName();

3-6-9 提取bean上的注解对象,获取注解中的属性值

                String jobClass = clazz.getName();
                String jobName = this.jobZookeeperProperties.getNamespace() + "." + conf.name();
                String cron = conf.cron();
                String shardingItemParameters = conf.shardingItemParameters();
                String description = conf.description();
                String jobParameter = conf.jobParameter();
                String jobExceptionHandler = conf.jobExceptionHandler();
                String executorServiceHandler = conf.executorServiceHandler();

                String jobShardingStrategyClass = conf.jobShardingStrategyClass();
                String eventTraceRdbDataSource = conf.eventTraceRdbDataSource();
                String scriptCommandLine = conf.scriptCommandLine();

                boolean failover = conf.failover();
                boolean misfire = conf.misfire();
                boolean overwrite = conf.overwrite();
                boolean disabled = conf.disabled();
                boolean monitorExecution = conf.monitorExecution();
                boolean streamingProcess = conf.streamingProcess();

                int shardingTotalCount = conf.shardingTotalCount();
                int monitorPort = conf.monitorPort();
                int maxTimeDiffSeconds = conf.maxTimeDiffSeconds();
                int reconcileIntervalMinutes = conf.reconcileIntervalMinutes();

3-6-10 接下来为了创建 LiteJobConfiguration ,依次需要 JobTypeConfiguration,JobCoreConfiguration

JobCoreConfiguration jobCoreConfiguration = JobCoreConfiguration.newBuilder(jobName, cron, shardingTotalCount)
                        .shardingItemParameters(shardingItemParameters)
                        .description(description)
                        .failover(failover)
                        .jobParameter(jobParameter)
                        .misfire(misfire)
                        .jobParameter(jobParameter)
                        .jobProperties(JobProperties.JobPropertiesEnum.JOB_EXCEPTION_HANDLER.getKey(), jobExceptionHandler)
                        .jobProperties(JobProperties.JobPropertiesEnum.EXECUTOR_SERVICE_HANDLER.getKey(), executorServiceHandler)
                        .build();
                JobTypeConfiguration typeConfig = null;
                if (ElasticJobTypeEnum.SIMPLE.getType().equals(jobTypeName)) {
                    typeConfig = new SimpleJobConfiguration(jobCoreConfiguration, jobClass);
                }
                if (ElasticJobTypeEnum.DATAFLOW.getType().equals(jobTypeName)) {
                    typeConfig = new DataflowJobConfiguration(jobCoreConfiguration, jobClass, streamingProcess);
                }
                if (ElasticJobTypeEnum.SCRIPT.getType().equals(jobTypeName)) {
                    typeConfig = new ScriptJobConfiguration(jobCoreConfiguration, scriptCommandLine);
                }

                LiteJobConfiguration liteJobConfiguration = LiteJobConfiguration.newBuilder(typeConfig)
                        .overwrite(overwrite)
                        .disabled(disabled)
                        .monitorPort(monitorPort)
                        .monitorExecution(monitorExecution)
                        .maxTimeDiffSeconds(maxTimeDiffSeconds)
                        .jobShardingStrategyClass(jobShardingStrategyClass)
                        .reconcileIntervalMinutes(reconcileIntervalMinutes)
                        .build();

其中由推断出的 jobTypeName 来选择创建 简单任务 数据流任务 脚本任务

                JobTypeConfiguration typeConfig = null;
                if (ElasticJobTypeEnum.SIMPLE.getType().equals(jobTypeName)) {
                    typeConfig = new SimpleJobConfiguration(jobCoreConfiguration, jobClass);
                }
                if (ElasticJobTypeEnum.DATAFLOW.getType().equals(jobTypeName)) {
                    typeConfig = new DataflowJobConfiguration(jobCoreConfiguration, jobClass, streamingProcess);
                }
                if (ElasticJobTypeEnum.SCRIPT.getType().equals(jobTypeName)) {
                    typeConfig = new ScriptJobConfiguration(jobCoreConfiguration, scriptCommandLine);
                }

3-6-11 接下来为了创建 SpringJobScheduler 需要获取它的 BeanDefinition,并设置属性和方法,以及模式(单例或者多例)

                BeanDefinitionBuilder factory = BeanDefinitionBuilder.rootBeanDefinition(SpringJobScheduler.class);
                factory.setInitMethodName("init");
                factory.setScope("prototype");

3-6-12 注入任务对象,此处排除了脚本任务,继续注入 注册中心 zookeeperRegistryCenter 和 任务配置 liteJobConfiguration 对象

                BeanDefinitionBuilder factory = BeanDefinitionBuilder.rootBeanDefinition(SpringJobScheduler.class);

                factory.setInitMethodName("init");
                factory.setScope("prototype");

                if (!ElasticJobTypeEnum.SCRIPT.getType().equals(jobTypeName)) {
                    factory.addConstructorArgValue(confBean);
                }

                factory.addConstructorArgValue(zookeeperRegistryCenter);
                factory.addConstructorArgValue(liteJobConfiguration);

3-6-13 配置事件追踪器(需要在注解中配置) 事件监听器

                if (StringUtils.hasText(eventTraceRdbDataSource)) {
                    BeanDefinitionBuilder rdbFactory = BeanDefinitionBuilder.rootBeanDefinition(JobEventConfiguration.class);
                    rdbFactory.addConstructorArgValue(eventTraceRdbDataSource);
                    factory.addConstructorArgValue(rdbFactory.getBeanDefinition());
                }

                List elasticJobListeners = getTargetElasticJobListeners(conf);
                factory.addConstructorArgValue(elasticJobListeners);

获取监听器

private List<BeanDefinition> getTargetElasticJobListeners(ElasticJobConfig conf) {
        ManagedList<BeanDefinition> result = new ManagedList<>(2);
        String listeners = conf.listener();
        if (StringUtils.hasText(listeners)) {
            BeanDefinitionBuilder factory = BeanDefinitionBuilder.rootBeanDefinition(listeners);
            factory.setScope("prototype");
            result.add(factory.getBeanDefinition());
        }

        String distributedListeners = conf.distributedListener();
        if (StringUtils.hasText(distributedListeners)) {
            BeanDefinitionBuilder factory = BeanDefinitionBuilder.rootBeanDefinition(distributedListeners);
            factory.setScope("prototype");
            factory.addConstructorArgValue(Long.valueOf(conf.startedTimeoutMilliseconds()));
            factory.addConstructorArgValue(Long.valueOf(conf.completedTimeoutMilliseconds()));
            result.add(factory.getBeanDefinition());
        }
        return result;
    }

3-6-12 将 SpringJobScheduler 注入到容器中,并执行init方法

                DefaultListableBeanFactory defaultListableBeanFactory = (DefaultListableBeanFactory) context.getAutowireCapableBeanFactory();

                String registerBeanName = conf.name() + "SpringJobScheduler";
                defaultListableBeanFactory.registerBeanDefinition(registerBeanName, factory.getBeanDefinition());
                SpringJobScheduler scheduler = (SpringJobScheduler) context.getBean(registerBeanName);
                scheduler.init();

评论

暂无评论