易盾服务启动手册

外观

antispam-business-video — 点播视频内容安全 ​

分层：业务层 ｜ 部署单元：8 个（k8s 实部署 6 个） ｜ 数据库：PostgreSQL（库 antispam） ｜ base image：混合 base-image:jdk8 / base-image:jdk17（tag base-250409-pg）

一、模块定位与架构 ​

模块是易盾「点播视频」内容安全检测的服务端：受理客户提交的点播视频，入库并调度底层 AI 视频检测集群完成机审，汇聚人工审核结果后回调客户，同时负责检测数据入库、归档与计量。它不自己做模型推理，而是把"提交 → 调度 → 结果回流 → 出口"这条链路编排可靠。

架构分层

  • 协议层：facade/http-check-api/controller/submit/*（/v2/video/submit、/v3/video/check、/v4/video/submit）、facade/http-api（回调/查询）、facade/dubbo-provider/facade/dubbo-check-provider
  • 逻辑层：facade/dubbo-check-provider/TargetProcessor（按 targetId 编排 9 类检测器）、service/business（VideoProxyService 代理 AI 集群）
  • 数据层：service/base+storage/consumer/*（VideoSubmitHandler 入库、CdcConsumer 消费 CDC 结果、ES 落库/回调/归档）、scheduler
  • 公共：common、domain、service、facade/{dubbo-api,dubbo-check-api}、async、console/api、file/doc、file/sql

关键数据流

  1. 客户提交 → AbstractSubmitController.submitPool（限流/缓冲池/重复检测）
  2. VideoCheckService.check Dubbo 预检 → Kafka Antispam_Video_Submit
  3. VideoSubmitHandler 入库 → VideoStartCheckService 调度 AI（VideoProxyService HTTP 调 /video/yidun/start|detect|recover）
  4. AI 结果经 CDC Antispam_Cdc_Video_New 回流 → CdcConsumer 分发
  5. ES 落库 / VideoCallbackHandler 回调 / VideoArchiveService 归档 / VideoBillService 计量

二、可部署服务清单 ​

部署单元目录artifactId镜像名端口服务类型独立 Dockerfile是否进 kubernetes.yml
facade/dubbo-providerantispam-business-video-facade-dubbo-providerantispam-business-video-dubbo-provider无 HTTP（Dubbo）Dubbo Provider（查询出口）有是
facade/dubbo-check-providerantispam-business-video-facade-dubbo-check-providerantispam-business-video-dubbo-check-provider无 HTTP（Dubbo）Dubbo Provider（预检编排）有是
facade/http-check-apiantispam-business-video-facade-http-check-apiantispam-business-video-http-check-apibase 8510 / private 8080HTTP 提交/检测有是
facade/http-apiantispam-business-video-facade-http-apiantispam-business-video-http-apibase 8500 / private 8080HTTP 回调/查询有是
storageantispam-business-video-storageantispam-business-video-storage无 HTTP（Kafka/CDC 消费）消费落库/回调/归档有是
schedulerantispam-business-video-schedulerantispam-business-video-scheduler无 HTTP（定时任务）Elastic-Job有是
asyncantispam-business-video-asyncantispam-business-video-async无 HTTP（Kafka 消费）异步消费有否
console/apiantispam-business-video-console-api——控制台后端骨架无（不构建）否

common、domain、service/base、service/business、service、facade/dubbo-api、facade/dubbo-check-api、facade、file 为非部署库模块。

三、同模块启动顺序 ​

  1. facade/dubbo-check-provider — TargetProcessor 预检编排能力注册 ZK
  2. facade/dubbo-provider — 查询出口 Dubbo 服务（VideoFacadeProvider）
  3. facade/http-check-api — 提交入口，预检走 Dubbo，调度依赖 AI 集群可用
  4. facade/http-api — 回调/查询入口，依赖同库 DAO 与 Dubbo 提供方
  5. storage — 消费 Antispam_Video_Submit 与 CDC Antispam_Cdc_Video_New，需 Kafka/DB/PG/ES/OSS 就绪
  6. scheduler — Elastic-Job 任务（VideoReviseTask 等），需 ZK 与 DB

理由：Dubbo 提供方先于消费方；storage 的 CDC 消费需 CDC 源（Kafka）先可用，否则启动期订阅失败。

四、逐服务启动逻辑 ​

video 全部启动类使用 inline @ComponentScan，无自定义 @Enable*AutoConfiguration 间接层；仅存在 SwaggerConfiguration 这类辅助配置类。

facade/http-check-api ​

  • 启动类：com.netease.is.antispam.business.video.facade.http.HttpCheckApplication（facade/http-check-api/.../HttpCheckApplication.java）
  • @SpringBootApplication（无 exclude）：@EnableBusinessClient、@EnableAspectJAutoProxy、@EnableRateLimiter、@EnableRecoverComponent、@EnableI18nComponent、@EnableApolloConfig、@ComponentScan({"com.netease.is.antispam.business.video"})
  • main()：无自定义
  • 钩子：ElasticRestSearchService @PostConstruct（service/base）、VideoIdCache / VideoMachineCheckCache / NosCdnService @PostConstruct
  • Apollo：app.id=antispam-video_http-check，namespaces=application,antispam-audiovideo_common

facade/http-api ​

  • 启动类：com.netease.is.antispam.business.video.facade.http.HttpApplication
  • @SpringBootApplication；@EnableBusinessClient、@EnableAspectJAutoProxy(proxyTargetClass=true)、@EnableTransactionManagement、@EnableCdcDataSubscribe、@EnableOssClient、@EnableRateLimiter、@EnableRecoverComponent、@EnableI18nComponent、@ComponentScan({"com.netease.is.antispam.business.video"})、@MapperScan("com.netease.is.antispam.business.video.dao")、@PropertySources({@PropertySource("classpath:video-oss.properties")})
  • main()：无自定义；Apollo：app.id=antispam-video_http

facade/dubbo-provider ​

  • 启动类：com.netease.is.antispam.business.video.facade.dubbo.DubboApplication
  • exclude：DataSourceAutoConfiguration
  • @EnableBusinessClient、@EnableAspectJAutoProxy(proxyTargetClass=true)、@EnableTransactionManagement、@EnableCdcDataSubscribe、@EnableOssClient、@EnableRecoverComponent、@ComponentScan({"com.netease.is.antispam.business.video"})、@MapperScan("...business.video.dao")、@PropertySources({video-oss.properties})
  • main()：无自定义；Apollo：app.id=antispam-video_dubbo

facade/dubbo-check-provider ​

  • 启动类：com.netease.is.antispam.business.video.facade.dubbo.check.CheckDubboApplication
  • @SpringBootApplication、@EnableBusinessClient、@EnableAspectJAutoProxy、@ComponentScan({"com.netease.is.antispam.business.video"})、@EnableRecoverComponent、@EnableApolloConfig
  • main()：无自定义；Apollo：app.id=antispam-video_dubbo-check

storage ​

  • 启动类：com.netease.is.antispam.business.video.storage.StorageApplication
  • @SpringBootApplication；@EnableBusinessClient、@EnableAspectJAutoProxy(proxyTargetClass=true)、@EnableTransactionManagement、@EnableCdcDataSubscribe、@EnableOssClient、@EnableRecoverComponent、@ComponentScan({"com.netease.is.antispam.business.video","com.netease.is.antispam.components.callback"})、@MapperScan("...business.video.dao")、@PropertySources({video-oss.properties})
  • main()：无自定义
  • 钩子：configuration/YidunQosApplication（doOffline() pause Kafka）、StorageSmartLifecycle implements SmartLifecycle（start() 遍历 List<WrapRecover> 逐个 WrapRecover::start，失败仅记日志）、多个 AbstractArchiveHandler / AbstractCallbackHandler / AbstractCensorStatHandler / AbstractRequestStatHandler @PostConstruct
  • Apollo：app.id=antispam-video_storage

scheduler ​

  • 启动类：com.netease.is.antispam.business.video.scheduler.SchedulerApplication
  • @SpringBootApplication；@EnableBusinessClient、@EnableAspectJAutoProxy(proxyTargetClass=true)、@EnableTransactionManagement、@EnableElasticJob、@EnableOssClient、@EnableCdcDataSubscribe、@EnableRecoverComponent、@EnableApolloConfig、@ComponentScan({"com.netease.is.antispam.business.video","com.netease.is.antispam.components.callback.clean","com.netease.is.antispam.components.callback.activeretry"})、@MapperScan("...business.video.dao")、@PropertySources({video-oss.properties})
  • main()：无自定义；钩子：YidunQosApplication、VideoReviseTask @PostConstruct
  • Apollo：app.id=antispam-video_scheduler

async ​

  • 启动类：com.netease.is.antispam.business.video.async.AsyncApplication
  • @SpringBootApplication；@EnableAspectJAutoProxy(proxyTargetClass=true)、@EnableOssClient、@EnableRecoverComponent、@ComponentScan({"com.netease.is.antispam.business.video"})、@PropertySources({video-oss.properties})
  • 钩子：com.netease.is.antispam.business.video.async.YidunQosApplication
  • 有 Dockerfile 但不在 kubernetes.yml；异步逻辑实际由 storage 承担

console/api ​

  • 启动类：com.netease.is.antispam.business.video.console.api.ConsoleApplication（SwaggerConfiguration）
  • 无 Dockerfile、不在 kubernetes.yml

共通：所有启动类 main() 无业务自定义逻辑，仅 SpringApplication.run。

五、启动前置依赖 ​

依赖配置键/地址阻塞 or 弱依赖配置文件
ZooKeeperzookeeper://zk-0.zookeeper.yidun-infra:2181，group /yidun/antispam/online-new/yidun-antispam-dubbo；zookeeper.config.root.path=/netease-antispam/online/config阻塞application-private.properties
PostgreSQLjdbc:postgresql://<PG_HOST>:5432/antispam?connectTimeout=60&socketTimeout=130&TimeZone=Asia/Shanghai（org.postgresql.Driver）阻塞（http-api/http-check-api/dubbo-provider/storage/scheduler）application-private.properties
Kafka / CDCnetease.kafka.producer.cluster.{main,log}.bootstrap.servers、cdc.kafka.bootstrap-servers=kafka-yidun-{0,1,2}.kafka-yidun-headless...:9092阻塞（storage/scheduler 启用 @EnableCdcDataSubscribe）application-private.properties
Redis Sentinelredis0-redis-ha.yidun-infra.svc:26379，master master01阻塞（限流/缓冲池）application-private.properties
视频 AI 集群video.check.host=http://video-service-api.yidun-ai.svc、area.ai.domain.map弱依赖（调度失败重试）application-private.properties
OSSclasspath:video-oss.properties（@PropertySource 显式引用）弱依赖video-oss.properties
Elasticsearchelasticsearch.cluster.servers.jd 等弱依赖application.properties
Apolloapp.id=antispam-video_*，apollo.bootstrap.namespaces=application,antispam-audiovideo_common，apollo.cache-dir=./apollo/cache阻塞application.properties
大区集群代理remote.clusterProxy.apiUrl=http://clusterfeature-yd.service.163.org/dataservice/cluster弱依赖application-private.properties

六、启动参数与 Profile ​

  • -Dspring.profiles.active：默认 dev；Dockerfile 与 k8s 覆盖为 private
  • Maven profile：buildAll.sh 以 -P private 全量构建后遍历 build.sh；模块 build.sh -e <env>
  • JAVA_OPTS（Dockerfile）：http-check-api/storage 等 jdk17 单元为 -Xmx1024m -Xms1024m -XX:+UseG1GC -Dspring.profiles.active=private；dubbo-provider（jdk8）为 -Xmx512m -Xms512m ...
  • JAVA_OPTS（k8s 覆写）：6 个 Deployment 统一 -Xmx1024m -Xms1024m -XX:+UseG1GC -Dspring.profiles.active=private（注意双空格），与 dubbo-provider Dockerfile 里的 512m 不一致
  • base image（混用）：facade/dubbo-provider、facade/dubbo-check-provider、scheduler 用 private-registry.nis.netease.com/library/base-image:jdk8；facade/http-check-api、facade/http-api、storage 用 base-image:jdk17；镜像 tag base-250409-pg
  • 镜像名三处：Dockerfile ENV APP_NAME=antispam-business-video-facade-<unit>、build.sh 推 private-registry.nis.netease.com/yidun/antispam-business-video-<unit>、k8s 拉 private-registry.yidun.internal/yidun/antispam-business-video-<unit>:1.0.0.private
  • k8s 通用：namespace yidun，replicas=2，env 注入 ENABLE_SKYWALKING=true、SW_AGENT_COLLECTOR_BACKEND_SERVICES=skywalking-skywalking-oap.yidun-infra.svc:11800

七、启动期踩坑 ​

  1. base image 混用 jdk8/jdk17：dubbo-provider/dubbo-check-provider/scheduler 仍是 base-image:jdk8，.jar 与依赖必须兼容 Java 8，新增 jdk17 专属 API 会在这三个单元启动即崩。
  2. 数据库是 PostgreSQL：org.postgresql.Driver + jdbc:postgresql://<PG_HOST>:5432/antispam；分页方言与 SQL（limit/offset、:: 转换）与其它模块的 MySQL/TiDB 不通用，迁移 SQL 时别照抄。
  3. @EnableCdcDataSubscribe 在 http-api/dubbo-provider/storage/scheduler 生效（与 text 模块相反），CDC 源是硬前置：cdc.kafka.bootstrap-servers 不可达会阻断启动。
  4. storage 的 StorageSmartLifecycle.start() 遍历 WrapRecover::start，内部 catch 后仅 log.error；isRunning() 恒返回 false，可能被重复触发，恢复器启动失败的信号容易被日志淹没。
  5. async 有独立 Dockerfile 却不在 kubernetes.yml；部署清单里看不到它，但异步消费逻辑实际被打包进 storage，别误以为异步链路缺失。
  6. video-oss.properties 用 @PropertySource 显式引用（非 application-*.properties 自动加载）；该文件缺失或未打进 jar 会直接启动失败。
  7. dubbo-provider Dockerfile JAVA_OPTS 是 -Xmx512m，k8s env 覆写成 -Xmx1024m；本地用 Dockerfile 默认值起会与线上内存口径不一致。
  8. 无任何自定义 @Enable*AutoConfiguration 间接层，扫包全部靠 inline @ComponentScan；漏写某个子包不会报错，只会静默少装配 bean（如某类 Consumer/Handler 不生效）。

八、跨模块前置 ​

  • antispam-business：TargetCache/ProductCache/SecretInfoCache/TargetConfigCache（@EnableBusinessClient）——判定 secretId/businessId/targetType/策略组的前置
  • antispam-image：VideoProxyService.getCheckParams 经 Dubbo 调 ImageCheckFacade 取检测参数；截图检测复用图片体系
  • antispam-video-solution：VideoProxyService HTTP 调 /video/yidun/start|detect|recover，实际机审由底层视频 AI 集群完成（video.check.host，可按 areaDomainMap 切海外大区）
  • antispam-list / antispam-rule / antispam-keyword：名单、规则、关键词检测 Dubbo 依赖
  • antispam-bill：VideoBillService 计量消息（antispam-bill-common/antispam-bill-domain）
  • antispam-cms：控制台经 VideoFacadeProvider 等 Dubbo 查询检测数据与操作记录
  • 基础设施：ZooKeeper、Kafka、Redis Sentinel、PostgreSQL、Elasticsearch、Apollo、OSS

相关：全局启动顺序 · 部署单元矩阵 · 踩坑速查

内部技术文档 · 请勿外传

网易易盾 · 反垃圾内容安全平台