SQL Injection (SQLi) remains a critical threat to web applications despite defensive mechanisms like WAF. Traditional tools rely on fixed payloads, limiting adaptability against obfuscated attacks. This paper proposes a checklist-integrated reinforcement learning framework that dynamically optimizes sqlmap configuration based on real-time feedback and discovered defense patterns. The framework comprises four core components: a Crawler for systematic reconnaissance, an RL agent that learns to configure sqlmap through Q-learning, Sqlmap for secure execution, and the target web application. The key innovation is encoding blocked keywords discovered during testing directly into the RL state space, enabling the agent to avoid redundant exploration of already-blocked patterns. Evaluated on SQLi MicroBenchmark (60 vulnerable and non-vulnerable SQL queries), the system achieves 53.3% true positive rate (16/30 vulnerable samples) with 904.9 average HTTP requests and 52.9 seconds average runtime in Feedback mode—a 60% reduction in requests and 64% improvement in runtime compared to sequential Sqlmap default. Perfect specificity (0% false positive rate) is maintained across 30 non-vulnerable samples. Payload analysis reveals that the agent learns diverse techniques including comment-based filtering bypass, parentheses escaping, time-based blind injection, and error-based exploitation. Results demonstrate that checklist-integrated RL reduces testing overhead while maintaining practical detection efficiency compared to traditional sequential scanning, and achieves competitive black-box performance without requiring database instrumentation. The approach offers practical value for automated security assessment in resource-constrained and time-sensitive penetration testing scenarios.