Skip to content

feat: LLM 미션 생성 응답 속도 개선 (thinking 비활성화 + flash-lite 전환) - #171

Merged
heeun98 merged 2 commits into
mainfrom
feature/PICK-247-llm-mission-generate
Jul 14, 2026
Merged

heeun98 merged 2 commits into
mainfrom
feature/PICK-247-llm-mission-generate

Conversation

@heeun98

@heeun98 heeun98 commented Jul 14, 2026

Copy link
Copy Markdown
Collaborator

🛠️ 작업 내용

  • Gemini thinking 비활성화: gemini-2.5 계열은 thinking(내부 추론)이 기본 활성화라 미션 1개 생성에도 최대 15초 지연 → 짧은 JSON 생성에 추론이 불필요하므로 generationConfig.thinkingConfig.thinkingBudget=0 적용
  • 모델을 gemini-2.5-flash-lite로 전환: 2.5 패밀리 중 지연시간·비용 최소, TTFT와 초당 토큰 생성 속도 우위. GEMINI_MODEL 환경변수로 배포 없이 되돌리기 가능 (기존 구조 동일)

🎯 리뷰 포인트

  • flash-lite의 생성 품질 트레이드오프: 프롬프트에 문장 패턴·예시가 촘촘해 짧은 미션 생성에는 충분할 것으로 예상하나, 배포 후 생성 문구 품질 확인 필요. 아쉬우면 운영 .env에 GEMINI_MODEL=gemini-2.5-flash 한 줄로 배포 없이 복귀 가능
  • 운영 .env에 GEMINI_MODEL이 설정돼 있으면 yml 기본값보다 우선하므로 배포 전 확인 필요

⏳ 작업 시간

  • 추정 시간:
  • 실제 시간:

heeun98 added 2 commits July 14, 2026 23:48
- gemini-2.5-flash는 thinking이 기본 활성화라 미션 1개 생성에도 수 초~15초 지연 발생
- 짧은 JSON 생성이라 추론이 불필요하므로 generationConfig.thinkingConfig.thinkingBudget=0으로 비활성화
- 2.5 패밀리 중 지연시간·비용 최소, TTFT와 초당 토큰 생성 속도 우위
- 미션 1개(짧은 JSON) 생성 태스크라 품질 요구가 낮아 lite로 충분
- GEMINI_MODEL 환경변수로 배포 없이 모델 교체 가능 (기존과 동일)
@coderabbitai

coderabbitai Bot commented Jul 14, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@heeun98, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 3 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: f07c8bbe-beb3-431e-acc0-03bf37a2a68c

📥 Commits

Reviewing files that changed from the base of the PR and between 52bf399 and 454f0db.

📒 Files selected for processing (2)
  • peektime-api/src/main/java/com/team/peektime_api/global/infra/llm/dto/GeminiRequest.java
  • peektime-api/src/main/resources/application.yml
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feature/PICK-247-llm-mission-generate

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@heeun98
heeun98 merged commit b3a4425 into main Jul 14, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant