Your Guide to Apache Dolphinschareder locally (without breaking things)

This article systematically determines the general process of preparing the local errors of Apache Dolphinschareder in IDEA, including environmental preparation, code formation, service start, and other basic steps to refer to it.

1. Prepare the basic component

1. JDK: v1.8.x (currently does not support JDK 11)  
2. Maven: v3.5+  
3. Node.js: v18.19.1+, install pnpm  
// Global installation  
npm install pnpm -g  
// Check registry  
pnpm config get registry  
// Switch to Taobao registry  
pnpm config set registry https://registry.npmmirror.com/  

4. Zookeeper: 3.6.3 (this version is used by big data platforms, DS reuses the platform's Zookeeper). When using the latest DS, it uses curator 5.3.0  
Curator 5.0 supports Zookeeper 3.6.X, no longer supports Zookeeper 3.4.X  
Curator 4.X supports Zookeeper 3.5.X, with soft compatibility for 3.4.X  
Curator 2.X supports Zookeeper 3.4.X  

5. MySQL version check:  
mysql> select version();  
+-----------+  
| version() |  
+-----------+  
| 5.7.44    |  
+-----------+  
1 row in set (0.00 sec)  

2. Preparing

2.1 Creating database

source /Users/xxx/IdeaProjects/dolphinscheduler/dolphinscheduler-dao/src/main/resources/sql/dolphinscheduler_mysql.sql;  

2.2 Common keys.

# Local directory for storing scripts  
data.basedir.path=/tmp/dolphinscheduler  

# Storage medium selection (e.g., HDFS); for resource center and tenant directories  
resource.storage.type=HDFS  

# Root directory of resource center  
resource.storage.upload.base.path=/dolphinscheduler  

# User for HDFS operations (typically hdfs user)  
resource.hdfs.root.user=hdfs  

# HDFS defaultFS (for HA mode, place core-site.xml and hdfs-site.xml in resources folder)  
resource.hdfs.fs.defaultFS=hdfs://10.253.26.85:8020  

# Development mode (recommended for easier troubleshooting)  
development.state=true  

# YARN port  
resource.manager.httpaddress.port=8088  

# For YARN HA, configure multiple IPs separated by commas  
yarn.resourcemanager.ha.rm.ids=  

# For single YARN, replace ds1 with YARN IP; leave unchanged for HA mode  
yarn.application.status.address=http://ds1:%s/ws/v1/cluster/apps/%s  

2.3 Application.yaml for each service

Note: Install the Zookeeper contact address mainly and the MySQL address (details have been deleted).

2.4 Logback-Spring.xml for each service

hiring To remove the console.

Here is the English translation:

3. The startup component

1) MasterServer:
Execute the main method of org.apache.dolphinscheduler.server.master.MasterServer in IntelliJ IDEA,
with VM Options:
-Dlogging.config=classpath:logback-spring.xml 
-Ddruid.mysql.usePingMethod=false 
-Dspring.profiles.active=mysql

2) WorkerServer:
Execute the main method of org.apache.dolphinscheduler.server.worker.WorkerServer in IntelliJ IDEA,
with VM Options:
-Dlogging.config=classpath:logback-spring.xml 
-Ddruid.mysql.usePingMethod=false 
-Dspring.profiles.active=mysql

3) ApiApplicationServer:
Execute the main method of org.apache.dolphinscheduler.api.ApiApplicationServer in IntelliJ IDEA,
with VM Options:
-Dlogging.config=classpath:logback-spring.xml 
-Dspring.profiles.active=api,mysql

After startup, you can browse OpenAPI documentation at:
http://localhost:12345/dolphinscheduler/swagger-ui/index.html

4) Frontend:
cd dolphinscheduler-ui
pnpm install
pnpm run dev

Error encountered:
qiaozhanwei@ dolphinscheduler-ui % pnpm run dev
> [email protected] dev /Users/qiaozhanwei/IdeaProjects/dolphinscheduler/dolphinscheduler-ui 
> vite

error when starting dev server:
Error: listen EADDRNOTAVAIL: address not available 192.168.1.4:5173
    at Server.setupListenHandle [as _listen2] (node:net:1313:21)
    at listenInCluster (node:net:1378:12)
    at GetAddrInfoReqWrap.doListen [as callback] (node:net:1516:7)
    at GetAddrInfoReqWrap.onlookup [as oncomplete] (node:dns:73:8)

Code modification:
On Mac, find IP address in terminal using command:
ipconfig getifaddr en0

After finding IP address, locate vite.config.ts file in project and modify as follows:

import { defineConfig } from 'vite'
import vue from '@vitejs/plugin-vue'

// https://vitejs.dev/config/
export default defineConfig({
    plugins: [vue()],
    base:'/',
    server:{
        host:'192.168.x.x',
        port:'5173',
        https:'false',
        open:'true',
        hmr:{
            protocol:'ws',
            host:'192.168.x.x'
        },
    }
})

Login URL:
http://10.x.x.x/login 
Use credentials admin/dolphinscheduler123 to login

4. 2.X version starting component

api server:
-Dlogging.config=classpath.logback-api.xml 
-Ddruid.mysql.usePingMethod=false 
-Dspring.profiles.active="default,api,mysql"

master:
-Dlogging.config=classpath.logback-master.xml 
-Ddruid.mysql.usePingMethod=false 
-Dspring.profiles.active="default,master,mysql"

worker:
-Dlogging.config=classpath.logback-worker.xml 
[Note:The worker configuration appears to be truncated in original text]