AI-Powered Website Scraper + Q&A

A full-stack AI-powered web application that scrapes website content and uses AI to answer user questions. Built with a modern monorepo architecture using Turborepo, featuring a Next.js frontend with Clerk authentication, Express API backend with BullMQ job processing, and PostgreSQL + Redis for data management. 

Workflow : URL + Question (Frontend) → API validates & saves to DB → BullMQ queue → Worker scrapes with Puppeteer → Groq AI processes → Answer saved to DB → Frontend displays result 

Key Features:

- 🏗️ Monorepo architecture (Turborepo)

- ⚛️ Frontend: Next.js 16 + TypeScript + Tailwind CSS

- 🔐 Authentication: Clerk

- 🚀 Backend: Express.js + TypeScript

- 📊 Database: Supabase PostgreSQL (Drizzle ORM)

- 🔄 Queue: BullMQ + Upstash Redis

- 🤖 AI: Groq Free AI Models

- 🕷️ Web Scraping: Puppeteer

- 📦 Deployment: Vercel (Frontend) + Railway (API)

Github 🔗 : https://github.com/IconicXDivyansh/Sbl.so-Intern-Assignment
Deployed 🔗 : https://sbl-so-intern-assignment-web-u8dl.vercel.app

03 Nov 2025

👀 Assignment details

Full Stack Developer Internship at Second Brain Labs

Development

Internship / Remote / Immediate JoiningStipend: ₹25–40k/month (based on skills)Read the JD here: Job DescriptionExperience: Minimum 1+ years of hands-on development experience📈 Internship will be convertible to full-time (2-3 months based on performance) Assignment OverviewYou’ll build a small...