Add test results and troubleshoot docs for scheduler-service
- test.md: 25 endpoint tests (13 GET + 12 POST) all PASS, including Kafka flow - troubleshoot.md: 9 real problems faced during deployment with fixes - Updated mkdocs.yml nav to include new pages Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
4e2ee8cd83
commit
7f7d5416dd
812
docs/test.md
Normal file
812
docs/test.md
Normal file
@ -0,0 +1,812 @@
|
|||||||
|
# Test Results — Scheduler Service
|
||||||
|
|
||||||
|
**Namespace:** `sagarpatil-4844`
|
||||||
|
**Base URL:** `https://scheduler-service-sagarpatil-4844.os-tf-qa.onlinesales.ai/schedulerService`
|
||||||
|
**Database:** `os_scheduling_db` on `mysql-8-0`
|
||||||
|
**Date:** 2026-03-20
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Seed Data
|
||||||
|
|
||||||
|
Data seeded from `testconfig/mysql/data.sql`:
|
||||||
|
- 15 status_types (SUBMITTED, RUNNING, COMPLETED, FAILED, etc.)
|
||||||
|
- 1 cluster: "Cluster 1" (id=1)
|
||||||
|
- 1 group: "os-group-1" (id=1)
|
||||||
|
- 1 job_type: "Default" for os-group-1
|
||||||
|
- 1 job_detail, 1 schedule, 1 job, 2 job_triggers, 1 job_stat
|
||||||
|
- 1 command_prefix_image_mapping, 1 container_image_version_tag, 1 tag
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## GET Endpoint Tests
|
||||||
|
|
||||||
|
### 1. GET /cluster?id=1
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl 'https://scheduler-service-sagarpatil-4844.os-tf-qa.onlinesales.ai/schedulerService/cluster?id=1'
|
||||||
|
```
|
||||||
|
|
||||||
|
**Response:**
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"clusters": [
|
||||||
|
{
|
||||||
|
"id": 1,
|
||||||
|
"name": "Cluster 1",
|
||||||
|
"description": "Test Cluster 1",
|
||||||
|
"status": 1,
|
||||||
|
"createdAt": 1742428200000,
|
||||||
|
"updatedAt": 1742428200000
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"status": "OK"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
**Result:** PASS
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 2. GET /cluster?name=Cluster 1
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl 'https://scheduler-service-sagarpatil-4844.os-tf-qa.onlinesales.ai/schedulerService/cluster?name=Cluster%201'
|
||||||
|
```
|
||||||
|
|
||||||
|
**Response:**
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"clusters": [
|
||||||
|
{
|
||||||
|
"id": 1,
|
||||||
|
"name": "Cluster 1",
|
||||||
|
"description": "Test Cluster 1",
|
||||||
|
"status": 1
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"status": "OK"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
**Result:** PASS
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 3. GET /group?id=1
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl 'https://scheduler-service-sagarpatil-4844.os-tf-qa.onlinesales.ai/schedulerService/group?id=1'
|
||||||
|
```
|
||||||
|
|
||||||
|
**Response:**
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"groupDetails": [
|
||||||
|
{
|
||||||
|
"id": 1,
|
||||||
|
"groupName": "os-group-1",
|
||||||
|
"clusterId": 1,
|
||||||
|
"status": 1,
|
||||||
|
"maxParallelJobs": 10,
|
||||||
|
"createdAt": 1742428200000,
|
||||||
|
"updatedAt": 1742428200000
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"status": "OK"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
**Result:** PASS
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 4. GET /group?groupName=os-group-1
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl 'https://scheduler-service-sagarpatil-4844.os-tf-qa.onlinesales.ai/schedulerService/group?groupName=os-group-1'
|
||||||
|
```
|
||||||
|
|
||||||
|
**Response:**
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"groupDetails": [
|
||||||
|
{
|
||||||
|
"id": 1,
|
||||||
|
"groupName": "os-group-1",
|
||||||
|
"clusterId": 1,
|
||||||
|
"status": 1,
|
||||||
|
"maxParallelJobs": 10
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"status": "OK"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
**Result:** PASS
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 5. GET /jobTrigger?id=1
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl 'https://scheduler-service-sagarpatil-4844.os-tf-qa.onlinesales.ai/schedulerService/jobTrigger?id=1'
|
||||||
|
```
|
||||||
|
|
||||||
|
**Response:**
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"jobTriggers": [
|
||||||
|
{
|
||||||
|
"id": 1,
|
||||||
|
"jobName": "TestJob",
|
||||||
|
"groupName": "os-group-1",
|
||||||
|
"applicationName": "TestApp",
|
||||||
|
"status": "SUBMITTED",
|
||||||
|
"clusterId": 1,
|
||||||
|
"createdAt": 1742428200000,
|
||||||
|
"updatedAt": 1742428200000
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"status": "OK"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
**Result:** PASS
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 6. GET /jobTrigger?jobName=TestJob
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl 'https://scheduler-service-sagarpatil-4844.os-tf-qa.onlinesales.ai/schedulerService/jobTrigger?jobName=TestJob'
|
||||||
|
```
|
||||||
|
|
||||||
|
**Response:**
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"jobTriggers": [
|
||||||
|
{
|
||||||
|
"id": 1,
|
||||||
|
"jobName": "TestJob",
|
||||||
|
"groupName": "os-group-1",
|
||||||
|
"applicationName": "TestApp",
|
||||||
|
"status": "SUBMITTED"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"status": "OK"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
**Result:** PASS
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 7. GET /schedules?id=1
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl 'https://scheduler-service-sagarpatil-4844.os-tf-qa.onlinesales.ai/schedulerService/schedules?id=1'
|
||||||
|
```
|
||||||
|
|
||||||
|
**Response:**
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"schedules": [
|
||||||
|
{
|
||||||
|
"id": 1,
|
||||||
|
"name": "DefaultTestSchedule",
|
||||||
|
"groupName": "os-group-1",
|
||||||
|
"cronExpression": "0 0/5 * * * ?",
|
||||||
|
"status": 1,
|
||||||
|
"createdAt": 1742428200000,
|
||||||
|
"updatedAt": 1742428200000
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"status": "OK"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
**Result:** PASS
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 8. GET /jobDetail?jobDetailId=1
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl 'https://scheduler-service-sagarpatil-4844.os-tf-qa.onlinesales.ai/schedulerService/jobDetail?jobDetailId=1'
|
||||||
|
```
|
||||||
|
|
||||||
|
**Response:**
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"jobDetails": [
|
||||||
|
{
|
||||||
|
"jobDetailId": 1,
|
||||||
|
"command": "echo hello",
|
||||||
|
"priority": 1,
|
||||||
|
"retryCount": 3,
|
||||||
|
"retryPeriod": 60,
|
||||||
|
"timeout": 3600,
|
||||||
|
"status": 1
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"status": "OK"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
**Result:** PASS
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 9. GET /jobType?groupName=os-group-1&name=Default
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl 'https://scheduler-service-sagarpatil-4844.os-tf-qa.onlinesales.ai/schedulerService/jobType?groupName=os-group-1&name=Default'
|
||||||
|
```
|
||||||
|
|
||||||
|
**Response:**
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"jobTypes": [
|
||||||
|
{
|
||||||
|
"id": 1,
|
||||||
|
"name": "Default",
|
||||||
|
"groupName": "os-group-1",
|
||||||
|
"containerImage": "default-image",
|
||||||
|
"status": 1
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"status": "OK"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
**Result:** PASS
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 10. GET /tags?id=1&groupId=1
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl 'https://scheduler-service-sagarpatil-4844.os-tf-qa.onlinesales.ai/schedulerService/tags?id=1&groupId=1'
|
||||||
|
```
|
||||||
|
|
||||||
|
**Response:**
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"tags": [
|
||||||
|
{
|
||||||
|
"id": 1,
|
||||||
|
"name": "testTag",
|
||||||
|
"groupId": 1,
|
||||||
|
"status": 1
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"status": "OK"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
**Result:** PASS
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 11. GET /commandPrefixImageMapping?command=python&groupName=os-group-1
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl 'https://scheduler-service-sagarpatil-4844.os-tf-qa.onlinesales.ai/schedulerService/commandPrefixImageMapping?command=python&groupName=os-group-1'
|
||||||
|
```
|
||||||
|
|
||||||
|
**Response:**
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"commandPrefixImageMappings": [
|
||||||
|
{
|
||||||
|
"id": 1,
|
||||||
|
"command": "python",
|
||||||
|
"groupName": "os-group-1",
|
||||||
|
"containerImage": "python:3.9",
|
||||||
|
"status": 1
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"status": "OK"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
**Result:** PASS
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 12. GET /containerImageVersionTag?containerImage=container-image-1
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl 'https://scheduler-service-sagarpatil-4844.os-tf-qa.onlinesales.ai/schedulerService/containerImageVersionTag?containerImage=container-image-1'
|
||||||
|
```
|
||||||
|
|
||||||
|
**Response:**
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"containerImageVersionTags": [
|
||||||
|
{
|
||||||
|
"id": 1,
|
||||||
|
"containerImage": "container-image-1",
|
||||||
|
"versionTag": "v1.0",
|
||||||
|
"status": 1
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"status": "OK"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
**Result:** PASS
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 13. GET /schedulesMetadata?scheduleId=1&groupId=1
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl 'https://scheduler-service-sagarpatil-4844.os-tf-qa.onlinesales.ai/schedulerService/schedulesMetadata?scheduleId=1&groupId=1'
|
||||||
|
```
|
||||||
|
|
||||||
|
**Response:**
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"schedulesMetadata": [
|
||||||
|
{
|
||||||
|
"id": 1,
|
||||||
|
"scheduleId": 1,
|
||||||
|
"groupId": 1,
|
||||||
|
"key": "testKey",
|
||||||
|
"value": "testValue",
|
||||||
|
"status": 1
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"status": "OK"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
**Result:** PASS
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## POST Endpoint Tests
|
||||||
|
|
||||||
|
### 14. POST /group
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -X POST 'https://scheduler-service-sagarpatil-4844.os-tf-qa.onlinesales.ai/schedulerService/group' \
|
||||||
|
-H 'Content-Type: application/json' \
|
||||||
|
-d '{
|
||||||
|
"groupDetail": {
|
||||||
|
"groupName": "os-group-2",
|
||||||
|
"clusterId": 1,
|
||||||
|
"status": 1,
|
||||||
|
"maxParallelJobs": 5
|
||||||
|
}
|
||||||
|
}'
|
||||||
|
```
|
||||||
|
|
||||||
|
**Response:**
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"status": "OK",
|
||||||
|
"groupDetail": {
|
||||||
|
"id": 2,
|
||||||
|
"groupName": "os-group-2",
|
||||||
|
"clusterId": 1,
|
||||||
|
"status": 1,
|
||||||
|
"maxParallelJobs": 5
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
**Result:** PASS
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 15. POST /schedules
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -X POST 'https://scheduler-service-sagarpatil-4844.os-tf-qa.onlinesales.ai/schedulerService/schedules' \
|
||||||
|
-H 'Content-Type: application/json' \
|
||||||
|
-d '{
|
||||||
|
"schedule": {
|
||||||
|
"name": "NewTestSchedule",
|
||||||
|
"groupName": "os-group-1",
|
||||||
|
"cronExpression": "0 0/10 * * * ?",
|
||||||
|
"status": 1
|
||||||
|
}
|
||||||
|
}'
|
||||||
|
```
|
||||||
|
|
||||||
|
**Response:**
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"status": "OK",
|
||||||
|
"schedule": {
|
||||||
|
"id": 2,
|
||||||
|
"name": "NewTestSchedule",
|
||||||
|
"groupName": "os-group-1",
|
||||||
|
"cronExpression": "0 0/10 * * * ?",
|
||||||
|
"status": 1
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
**Result:** PASS
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 16. POST /jobTrigger
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -X POST 'https://scheduler-service-sagarpatil-4844.os-tf-qa.onlinesales.ai/schedulerService/jobTrigger' \
|
||||||
|
-H 'Content-Type: application/json' \
|
||||||
|
-d '{
|
||||||
|
"jobTrigger": {
|
||||||
|
"id": 1,
|
||||||
|
"status": "SUBMITTED"
|
||||||
|
}
|
||||||
|
}'
|
||||||
|
```
|
||||||
|
|
||||||
|
**Response:**
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"status": "OK",
|
||||||
|
"jobTrigger": {
|
||||||
|
"id": 1,
|
||||||
|
"status": "SUBMITTED"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
**Result:** PASS
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 17. POST /jobStat
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -X POST 'https://scheduler-service-sagarpatil-4844.os-tf-qa.onlinesales.ai/schedulerService/jobStat' \
|
||||||
|
-H 'Content-Type: application/json' \
|
||||||
|
-d '{
|
||||||
|
"jobStat": {
|
||||||
|
"jobName": "TestJob",
|
||||||
|
"groupName": "os-group-1",
|
||||||
|
"applicationName": "TestApp",
|
||||||
|
"status": "COMPLETED",
|
||||||
|
"startTime": 1742428200000,
|
||||||
|
"endTime": 1742431800000
|
||||||
|
}
|
||||||
|
}'
|
||||||
|
```
|
||||||
|
|
||||||
|
**Response:**
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"status": "OK",
|
||||||
|
"jobStat": {
|
||||||
|
"id": 2,
|
||||||
|
"jobName": "TestJob",
|
||||||
|
"groupName": "os-group-1",
|
||||||
|
"status": "COMPLETED"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
**Result:** PASS
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 18. POST /tags
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -X POST 'https://scheduler-service-sagarpatil-4844.os-tf-qa.onlinesales.ai/schedulerService/tags' \
|
||||||
|
-H 'Content-Type: application/json' \
|
||||||
|
-d '{
|
||||||
|
"tag": {
|
||||||
|
"name": "newTag",
|
||||||
|
"groupId": 1,
|
||||||
|
"status": 1
|
||||||
|
}
|
||||||
|
}'
|
||||||
|
```
|
||||||
|
|
||||||
|
**Response:**
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"status": "OK",
|
||||||
|
"tag": {
|
||||||
|
"id": 2,
|
||||||
|
"name": "newTag",
|
||||||
|
"groupId": 1,
|
||||||
|
"status": 1
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
**Result:** PASS
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 19. POST /containerImageVersionTag
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -X POST 'https://scheduler-service-sagarpatil-4844.os-tf-qa.onlinesales.ai/schedulerService/containerImageVersionTag' \
|
||||||
|
-H 'Content-Type: application/json' \
|
||||||
|
-d '{
|
||||||
|
"containerImageVersionTag": {
|
||||||
|
"containerImage": "container-image-2",
|
||||||
|
"versionTag": "v2.0",
|
||||||
|
"status": 1
|
||||||
|
}
|
||||||
|
}'
|
||||||
|
```
|
||||||
|
|
||||||
|
**Response:**
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"status": "OK",
|
||||||
|
"containerImageVersionTag": {
|
||||||
|
"id": 2,
|
||||||
|
"containerImage": "container-image-2",
|
||||||
|
"versionTag": "v2.0",
|
||||||
|
"status": 1
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
**Result:** PASS
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 20. POST /schedulesMetadata
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -X POST 'https://scheduler-service-sagarpatil-4844.os-tf-qa.onlinesales.ai/schedulerService/schedulesMetadata' \
|
||||||
|
-H 'Content-Type: application/json' \
|
||||||
|
-d '{
|
||||||
|
"schedulesMetadata": {
|
||||||
|
"scheduleId": 1,
|
||||||
|
"groupId": 1,
|
||||||
|
"key": "newKey",
|
||||||
|
"value": "newValue",
|
||||||
|
"status": 1
|
||||||
|
}
|
||||||
|
}'
|
||||||
|
```
|
||||||
|
|
||||||
|
**Response:**
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"status": "OK",
|
||||||
|
"schedulesMetadata": {
|
||||||
|
"id": 2,
|
||||||
|
"scheduleId": 1,
|
||||||
|
"groupId": 1,
|
||||||
|
"key": "newKey",
|
||||||
|
"value": "newValue",
|
||||||
|
"status": 1
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
**Result:** PASS
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 21. POST /commandPrefixImageMapping
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -X POST 'https://scheduler-service-sagarpatil-4844.os-tf-qa.onlinesales.ai/schedulerService/commandPrefixImageMapping' \
|
||||||
|
-H 'Content-Type: application/json' \
|
||||||
|
-d '{
|
||||||
|
"commandPrefixImageMapping": {
|
||||||
|
"command": "java",
|
||||||
|
"groupName": "os-group-1",
|
||||||
|
"containerImage": "openjdk:11",
|
||||||
|
"status": 1
|
||||||
|
}
|
||||||
|
}'
|
||||||
|
```
|
||||||
|
|
||||||
|
**Response:**
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"status": "OK",
|
||||||
|
"commandPrefixImageMapping": {
|
||||||
|
"id": 2,
|
||||||
|
"command": "java",
|
||||||
|
"groupName": "os-group-1",
|
||||||
|
"containerImage": "openjdk:11",
|
||||||
|
"status": 1
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
**Result:** PASS
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 22. POST /jobType
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -X POST 'https://scheduler-service-sagarpatil-4844.os-tf-qa.onlinesales.ai/schedulerService/jobType' \
|
||||||
|
-H 'Content-Type: application/json' \
|
||||||
|
-d '{
|
||||||
|
"jobType": {
|
||||||
|
"name": "CustomType",
|
||||||
|
"groupName": "os-group-1",
|
||||||
|
"containerImage": "custom-image:latest",
|
||||||
|
"monitoringTags": "{\"FAILED_JOBS\": \"sev4\"}",
|
||||||
|
"status": 1
|
||||||
|
}
|
||||||
|
}'
|
||||||
|
```
|
||||||
|
|
||||||
|
**Response:**
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"status": "OK",
|
||||||
|
"jobType": {
|
||||||
|
"id": 2,
|
||||||
|
"name": "CustomType",
|
||||||
|
"groupName": "os-group-1",
|
||||||
|
"containerImage": "custom-image:latest",
|
||||||
|
"monitoringTags": "{\"FAILED_JOBS\": \"sev4\"}",
|
||||||
|
"status": 1
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
**Result:** PASS
|
||||||
|
|
||||||
|
> **Note:** `monitoringTags` must be a JSON **string** (escaped), not a raw JSON object.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 23. POST /jobDetail
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -X POST 'https://scheduler-service-sagarpatil-4844.os-tf-qa.onlinesales.ai/schedulerService/jobDetail' \
|
||||||
|
-H 'Content-Type: application/json' \
|
||||||
|
-d '{
|
||||||
|
"jobDetail": {
|
||||||
|
"jobDetailId": 1,
|
||||||
|
"priority": 2,
|
||||||
|
"retryPeriod": 120
|
||||||
|
}
|
||||||
|
}'
|
||||||
|
```
|
||||||
|
|
||||||
|
**Response:**
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"status": "OK",
|
||||||
|
"jobDetail": {
|
||||||
|
"jobDetailId": 1,
|
||||||
|
"priority": 2,
|
||||||
|
"retryPeriod": 120
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
**Result:** PASS
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 24. POST /job (useArgoWorkflows: false)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -X POST 'https://scheduler-service-sagarpatil-4844.os-tf-qa.onlinesales.ai/schedulerService/job' \
|
||||||
|
-H 'Content-Type: application/json' \
|
||||||
|
-H 'os-scheduling-group: os-group-1' \
|
||||||
|
-d '{
|
||||||
|
"job": {
|
||||||
|
"jobName": "testJob1",
|
||||||
|
"applicationName": "testApp1",
|
||||||
|
"command": "echo hello",
|
||||||
|
"jobTypeName": "Default",
|
||||||
|
"useArgoWorkflows": false
|
||||||
|
}
|
||||||
|
}'
|
||||||
|
```
|
||||||
|
|
||||||
|
**Response:**
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"status": "OK",
|
||||||
|
"job": {
|
||||||
|
"jobId": 3,
|
||||||
|
"jobName": "testJob1",
|
||||||
|
"applicationName": "testApp1",
|
||||||
|
"status": "SUBMITTED"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
**Result:** PASS
|
||||||
|
|
||||||
|
> Creates 3 DB entries: `job_details`, `jobs`, `job_triggers`. No Kafka event published.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 25. POST /job (useArgoWorkflows: true) — Kafka Flow
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -X POST 'https://scheduler-service-sagarpatil-4844.os-tf-qa.onlinesales.ai/schedulerService/job' \
|
||||||
|
-H 'Content-Type: application/json' \
|
||||||
|
-H 'os-scheduling-group: os-group-1' \
|
||||||
|
-d '{
|
||||||
|
"job": {
|
||||||
|
"jobName": "argoJob1",
|
||||||
|
"applicationName": "testApp1",
|
||||||
|
"command": "echo argo",
|
||||||
|
"jobTypeName": "Default",
|
||||||
|
"useArgoWorkflows": true
|
||||||
|
}
|
||||||
|
}'
|
||||||
|
```
|
||||||
|
|
||||||
|
**Response:**
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"status": "OK",
|
||||||
|
"job": {
|
||||||
|
"jobId": 6,
|
||||||
|
"jobName": "argoJob1",
|
||||||
|
"applicationName": "testApp1",
|
||||||
|
"status": "SUBMITTED"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Scheduler Logs (confirmed):**
|
||||||
|
```
|
||||||
|
ArgoEventSender: Sending event for job 6 | eventType: argo_ondemand_cluster_1 | eventName: ARGO_JOB_SUBMISSION
|
||||||
|
ArgoEventSender: Successfully posted event for job: 6
|
||||||
|
```
|
||||||
|
|
||||||
|
**Result:** PASS
|
||||||
|
|
||||||
|
> Full flow: `POST /job` → `ArgoEventSender` → `EventManager` → Kafka (`GCP_KAFKA`)
|
||||||
|
> Requires `PROD_KAFKA_TOPIC_VERSION_SETTING` in property-settings DB.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## POST-Only Endpoints (No GET)
|
||||||
|
|
||||||
|
| Endpoint | GET | POST | Reason |
|
||||||
|
|----------|-----|------|--------|
|
||||||
|
| `/job` | Not supported (500) | PASS | `initServletObjectMap(null, JobPostRequest.class, ...)` |
|
||||||
|
| `/jobStat` | Not supported (500) | PASS | `initServletObjectMap(null, JobStatPostRequest.class, ...)` |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
|
||||||
|
### Test Results
|
||||||
|
|
||||||
|
| # | Endpoint | Method | Result |
|
||||||
|
|---|----------|--------|--------|
|
||||||
|
| 1 | /cluster?id=1 | GET | PASS |
|
||||||
|
| 2 | /cluster?name=Cluster 1 | GET | PASS |
|
||||||
|
| 3 | /group?id=1 | GET | PASS |
|
||||||
|
| 4 | /group?groupName=os-group-1 | GET | PASS |
|
||||||
|
| 5 | /jobTrigger?id=1 | GET | PASS |
|
||||||
|
| 6 | /jobTrigger?jobName=TestJob | GET | PASS |
|
||||||
|
| 7 | /schedules?id=1 | GET | PASS |
|
||||||
|
| 8 | /jobDetail?jobDetailId=1 | GET | PASS |
|
||||||
|
| 9 | /jobType?groupName=...&name=... | GET | PASS |
|
||||||
|
| 10 | /tags?id=1&groupId=1 | GET | PASS |
|
||||||
|
| 11 | /commandPrefixImageMapping | GET | PASS |
|
||||||
|
| 12 | /containerImageVersionTag | GET | PASS |
|
||||||
|
| 13 | /schedulesMetadata | GET | PASS |
|
||||||
|
| 14 | /group | POST | PASS |
|
||||||
|
| 15 | /schedules | POST | PASS |
|
||||||
|
| 16 | /jobTrigger | POST | PASS |
|
||||||
|
| 17 | /jobStat | POST | PASS |
|
||||||
|
| 18 | /tags | POST | PASS |
|
||||||
|
| 19 | /containerImageVersionTag | POST | PASS |
|
||||||
|
| 20 | /schedulesMetadata | POST | PASS |
|
||||||
|
| 21 | /commandPrefixImageMapping | POST | PASS |
|
||||||
|
| 22 | /jobType | POST | PASS |
|
||||||
|
| 23 | /jobDetail | POST | PASS |
|
||||||
|
| 24 | /job (useArgoWorkflows: false) | POST | PASS |
|
||||||
|
| 25 | /job (useArgoWorkflows: true) | POST | PASS |
|
||||||
|
|
||||||
|
**Total: 25/25 PASS**
|
||||||
|
|
||||||
|
### Key Findings
|
||||||
|
|
||||||
|
- **13 GET endpoints** all working with correct query parameters
|
||||||
|
- **9 POST endpoints** all working with correct JSON body format
|
||||||
|
- **2 POST-only endpoints** (`/job`, `/jobStat`) — GET returns 500 by design
|
||||||
|
- **Kafka flow verified** — `useArgoWorkflows: true` publishes events via EventManager to Kafka
|
||||||
|
- **Event type:** `argo_ondemand_cluster_1`, **Event name:** `ARGO_JOB_SUBMISSION`
|
||||||
|
- **Application:** `osOnDemandJobsArgoRunner`
|
||||||
|
|
||||||
|
### Dependencies Verified
|
||||||
|
|
||||||
|
| Dependency | Status | Notes |
|
||||||
|
|------------|--------|-------|
|
||||||
|
| MySQL (os_scheduling_db) | Working | Via Hibernate XML, no Hades needed |
|
||||||
|
| Event-Manager-V2 | Working | Receives Argo events from scheduler |
|
||||||
|
| Property-Settings-V2 | Working | Provides PROD_KAFKA_TOPIC_VERSION_SETTING |
|
||||||
|
| Kafka | Working | Receives events via GCP_KAFKA path |
|
||||||
|
| Hades | Not needed | Scheduler uses direct Hibernate connection |
|
||||||
192
docs/troubleshoot.md
Normal file
192
docs/troubleshoot.md
Normal file
@ -0,0 +1,192 @@
|
|||||||
|
# Troubleshoot Guide — Scheduler Service
|
||||||
|
|
||||||
|
All real problems faced during deployment and testing at namespace `sagarpatil-4844` on `os-tf-qa.onlinesales.ai`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Problem 1: Hades Context Not Needed
|
||||||
|
|
||||||
|
**Symptom:** Init container `setup-hades-context` was failing/hanging because Hades service was not available or not needed.
|
||||||
|
|
||||||
|
**Root Cause:** Scheduler-service does NOT use Hades for DB credentials. It connects to MySQL directly via Hibernate XML config (`dbConfig.hibernateXml`). No code in `BaseCommunicator.java` calls `HadesClient.getContext()` — it only calls `HibernateDatabase.initHibernateDB(filePath)`.
|
||||||
|
|
||||||
|
**Fix:**
|
||||||
|
- Set `hadesContext.enabled: false` in both `values.yaml` and `values-local.yaml`
|
||||||
|
- Removed Hades from `catalog-info.yaml` dependency-urls and `spec.dependsOn`
|
||||||
|
- Removed hades-service from `externalServices` in values files
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Problem 2: GET /job and GET /jobStat Return "Internal Error"
|
||||||
|
|
||||||
|
**Symptom:**
|
||||||
|
```
|
||||||
|
curl .../schedulerService/job?jsonQuery={"id":1}
|
||||||
|
→ HTTP 500 "Internal Error"
|
||||||
|
```
|
||||||
|
|
||||||
|
**Root Cause:** These are POST-only endpoints. The servlet initialization uses `null` for GET class:
|
||||||
|
```java
|
||||||
|
// Job.java
|
||||||
|
initServletObjectMap(null, JobPostRequest.class, null, JobPostResponse.class);
|
||||||
|
// JobStat.java
|
||||||
|
initServletObjectMap(null, JobStatPostRequest.class, ...);
|
||||||
|
```
|
||||||
|
|
||||||
|
**Fix:** No fix needed — this is by design. Use POST for `/job` and `/jobStat`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Problem 3: GET /jobType Fails Without Both Parameters
|
||||||
|
|
||||||
|
**Symptom:**
|
||||||
|
```
|
||||||
|
curl .../schedulerService/jobType?groupName=os-group-1
|
||||||
|
→ empty or error response
|
||||||
|
```
|
||||||
|
|
||||||
|
**Root Cause:** `/jobType` GET requires BOTH `groupName` AND `name` (or `id`). A single param is not enough.
|
||||||
|
|
||||||
|
**Fix:**
|
||||||
|
```
|
||||||
|
GET /schedulerService/jobType?groupName=os-group-1&name=Default
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Problem 4: GET /tags Fails Without Required Parameters
|
||||||
|
|
||||||
|
**Symptom:**
|
||||||
|
```
|
||||||
|
curl .../schedulerService/tags?groupName=os-group-1
|
||||||
|
→ error response
|
||||||
|
```
|
||||||
|
|
||||||
|
**Root Cause:** `/tags` GET requires `id` (or `name`) AND `groupId`. Just `groupName` alone won't work.
|
||||||
|
|
||||||
|
**Fix:**
|
||||||
|
```
|
||||||
|
GET /schedulerService/tags?id=1&groupId=1
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Problem 5: POST /job with useArgoWorkflows:true Fails
|
||||||
|
|
||||||
|
**Symptom:**
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"status": "ERROR",
|
||||||
|
"message": "Failed to publish event to EventManager for job 2"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Root Cause:** Event-manager-v2 servlet initialization was crashing at `TopicSettingServlet.refreshTopicSettings()`. The flow:
|
||||||
|
1. Event-manager fetches Kafka creds from Hades — `{"brokerList":"kafka:9092","isAuthEnabled":false}` — OK
|
||||||
|
2. Event-manager calls property-settings for `PROD_KAFKA_TOPIC_VERSION_SETTING` with `entityType=AGENCY, entityValue=ALL`
|
||||||
|
3. Property-settings returns `{"properties":[null]}` — no matching property existed in DB
|
||||||
|
|
||||||
|
**Fix:** Insert the seed data into `property_settings_db_v2.properties` table:
|
||||||
|
```sql
|
||||||
|
INSERT IGNORE INTO properties (entity_type, entity_value, property_type, property_value, is_active)
|
||||||
|
VALUES ('AGENCY', 'ALL', 'PROD_KAFKA_TOPIC_VERSION_SETTING', '{"events":["GCP_KAFKA"]}', 1);
|
||||||
|
```
|
||||||
|
|
||||||
|
**Permanent Fix:** Updated `002_populate_properties_table.sql` in the `propertysettings-v2-chart` from `'{}'` to `'{"events":["GCP_KAFKA"]}'`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Problem 6: Wrong PROD_KAFKA_TOPIC_VERSION_SETTING Format
|
||||||
|
|
||||||
|
**Symptom:**
|
||||||
|
```
|
||||||
|
MismatchedInputException: Cannot deserialize value of type HashSet<KafkaVersion> from Object value
|
||||||
|
```
|
||||||
|
|
||||||
|
**Root Cause:** First attempt used wrong JSON format:
|
||||||
|
```json
|
||||||
|
{"topicVersionSettings":{"default":{"topic":"events","version":"v1"}}}
|
||||||
|
```
|
||||||
|
|
||||||
|
The `PropertySetttingUtil.java` expects `Map<String, Set<KafkaVersion>>`:
|
||||||
|
```java
|
||||||
|
TypeReference<Map<String, Set<KafkaVersion>>> typeRefence = new TypeReference<>(){};
|
||||||
|
```
|
||||||
|
|
||||||
|
**Fix:** Correct format is:
|
||||||
|
```json
|
||||||
|
{"events":["GCP_KAFKA"]}
|
||||||
|
```
|
||||||
|
|
||||||
|
Where `KafkaVersion` enum values are: `AWS_KAFKA`, `GCP_KAFKA`. Only `GCP_KAFKA` is active in `DbEvent.java` (AWS_KAFKA case is commented out).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Problem 7: POST /jobType monitoringTags Format
|
||||||
|
|
||||||
|
**Symptom:**
|
||||||
|
```
|
||||||
|
Marshalling Error when sending monitoringTags as JSON array
|
||||||
|
```
|
||||||
|
|
||||||
|
**Root Cause:** `monitoringTags` field expects a JSON **string** (stringified JSON), not a raw JSON object/array.
|
||||||
|
|
||||||
|
**Fix:**
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"jobType": {
|
||||||
|
"monitoringTags": "{\"FAILED_JOBS\": \"sev4\"}"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
Note the escaped quotes — the value is a string containing JSON, not a nested JSON object.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Problem 8: POST /job with useArgoWorkflows:false Works But true Fails
|
||||||
|
|
||||||
|
**Symptom:** Job creation itself works fine (creates `job_details`, `jobs`, `job_triggers` in DB) but the Kafka event publishing step fails.
|
||||||
|
|
||||||
|
**Root Cause:** When `useArgoWorkflows: true`, scheduler calls `ArgoEventSender` which POSTs to event-manager-v2. If event-manager's Kafka producer is not initialized (due to Problem 5), the event publish fails.
|
||||||
|
|
||||||
|
**Fix:** Fix Problem 5 first. Once event-manager has the correct `PROD_KAFKA_TOPIC_VERSION_SETTING`, the full flow works:
|
||||||
|
```
|
||||||
|
POST /job → ArgoEventSender → EventManager → Kafka (GCP_KAFKA)
|
||||||
|
```
|
||||||
|
|
||||||
|
Scheduler logs on success:
|
||||||
|
```
|
||||||
|
ArgoEventSender: Sending event for job 6 | eventType: argo_ondemand_cluster_1 | eventName: ARGO_JOB_SUBMISSION
|
||||||
|
ArgoEventSender: Successfully posted event for job: 6
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Problem 9: MySQL Connection — Finding Root Password
|
||||||
|
|
||||||
|
**Symptom:** Need to connect to `mysql-8-0` pod for debugging but don't know the root password.
|
||||||
|
|
||||||
|
**Fix:** Password is stored in Kubernetes secret:
|
||||||
|
```bash
|
||||||
|
kubectl get secret mysql-8-0 -n sagarpatil-4844 -o jsonpath='{.data.mysql-root-password}' | base64 -d
|
||||||
|
```
|
||||||
|
Result: `local-root-pass`
|
||||||
|
|
||||||
|
Connect:
|
||||||
|
```bash
|
||||||
|
kubectl exec -it mysql-8-0-0 -n sagarpatil-4844 -- mysql -uroot -plocal-root-pass
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Quick Reference
|
||||||
|
|
||||||
|
| Problem | Root Cause | Fix |
|
||||||
|
|---------|-----------|-----|
|
||||||
|
| Hades init hanging | Scheduler doesn't use Hades | Disable `hadesContext.enabled` |
|
||||||
|
| GET /job returns 500 | POST-only endpoint | Use POST instead |
|
||||||
|
| GET /jobType empty | Needs both params | Add `groupName` AND `name` |
|
||||||
|
| GET /tags fails | Needs `id` + `groupId` | Add both params |
|
||||||
|
| useArgoWorkflows fails | Missing Kafka topic setting | Seed `PROD_KAFKA_TOPIC_VERSION_SETTING` |
|
||||||
|
| Wrong topic format | Needs `Map<String, Set<KafkaVersion>>` | Use `{"events":["GCP_KAFKA"]}` |
|
||||||
|
| monitoringTags error | Must be stringified JSON | Escape quotes in string |
|
||||||
@ -7,6 +7,8 @@ nav:
|
|||||||
- API Reference: api-reference.md
|
- API Reference: api-reference.md
|
||||||
- Configuration: configuration.md
|
- Configuration: configuration.md
|
||||||
- Deployment: deployment.md
|
- Deployment: deployment.md
|
||||||
|
- Test Results: test.md
|
||||||
|
- Troubleshoot: troubleshoot.md
|
||||||
|
|
||||||
plugins:
|
plugins:
|
||||||
- techdocs-core
|
- techdocs-core
|
||||||
|
|||||||
Loading…
x
Reference in New Issue
Block a user