diff --git a/assignments/03-express-middleware.md b/assignments/03-express-middleware.md new file mode 100644 index 0000000..57eecd6 --- /dev/null +++ b/assignments/03-express-middleware.md @@ -0,0 +1,627 @@ +# **Assignment 3 — Extending Your Express App, and a Middleware Debugging Exercise** + +This assignment is to be done in the node-homework folder. Within that folder, create an `assignment3` ```git branch``` for your work. As you work on this assignment, add and commit your work to this branch periodically. + +> REMEMBER: Commit messages should be meaningful. `Week 3 assignment` is not a meaningful commit message. + +Your homework repo uses **ECMAScript modules** (`"type": "module"` in `package.json`): use `import` / `export` and include the `.js` extension on relative import paths (for example `./controllers/userController.js`). Automated tests use **Vitest** (invoked via the provided npm scripts, such as `npm run tdd`). + +## **Task 1: A Route Handler for User Registration, Logon, and Logoff** + +You have started work on the application you'll use for your final project. You now start adding the main functions. + +For your final project, you'll have users with todo lists. A user will be able to register with the application, log on, and create, modify, and delete tasks in their todo lists. You'll now create the route that does the register. That's a POST operation for the `/api/users/register` path. Add that to app.js, before the 404 handler. For now, you can just have it return a message. By convention, REST API routes start with `/api'. + +You cannot test this with the browser. Browsers send GET requests, and only do POSTs from within forms. Postman is the tool you'll use. Start it up. On the upper left-hand side, you see a `new` button. Create a new collection, called `node-homework`. On the upper right-hand side, you see an icon that is a rectangle with a little eye. No, it doesn't mean the Illuminati. This is the Postman environment. Create an environment variable called host, with a value of `http://localhost:3000`. This is the base URL for your requests. When it comes time to test your application as it is deployed on the internet, you can just change this environment variable. + +Hover over the node-homework collection and you'll see three dots. Click on those, and select 'add request'. Give it a name, perhaps `register`. A new request, by default, is a GET, but there is a pulldown to switch it to POST. Save the request, and then send it. If your Express app is running, you should see your message come back. Of course, to create a user record, you need data in the body of the request. So, click on the body tab for the request. Select the `raw` option. There's a pulldown to the right that says `Text`. Click on that, and choose the JSON option. Then, put JSON data in for the user you want to create. You need a name, an email, and a password. Remember that this is JSON, not a JavaScript object, so you have to have double quotes around the attribute names and string values. Save the request again, and then send it. The result is the same of course -- the request handler doesn't do more than send a message at the moment. + +Go back to app.js. You need to be able to get the body of the request. For that you need middleware, in this case middleware that Express provides. Add this line above your other routes: + +```js +app.use(express.json({ limit: "1kb" })); +``` + +This tells Express to parse JSON request bodies as they come in. The express.json() middleware only parses the request body if the Content-Type header says "application/json". The resulting object is stored in req.body. Of course, any routes that need to look at the request body have to come after this app.use(). + +Make the following change to the request handler: + +```js +app.post("/api/users/register", (req, res)=>{ + console.log("This data was posted", JSON.stringify(req.body)); + res.send("parsed the data"); +}); +``` + +Then try the Postman request again. You see the body in your server log, but you are still just sending back a message. + +What you should do for this request is store the user record. Eventually you'll store it in a database, but we haven't learned how to do that yet. So, for the moment, you can just store it in memory. Use the following globals: + +```js +global.user_id // The logged on user. This will be undefined or null if no user is logged on. +global.users // an array of user objects, initially empty. +global.tasks // an array of task object, initially empty. +``` + +Near the start of `app.js`, add: + +```js +global.user_id = null; +global.users = []; +global.tasks = []; +``` + +And then, change the app.post() as follows: + +```js +app.post("/api/users/register", (req, res)=>{ + const newUser = {...req.body}; // this makes a copy + global.users.push(newUser); + global.user_id = newUser; // After the registration step, the user is set to logged on. + delete req.body.password; + res.status(201).json(req.body); +}); +``` + + +When creating a new record, it is standard practice to return the object just created, but of course, you don't want to send back the user password. + +Test this with your Postman request. + +### **Why the Memory Store is Crude** + +Let's list all the hokey things you just did. + +1. There is no validation. You don't know if there was a valid body. Hopefully your Postman request did send one. + +2. You stored to memory (globals). When you restart the server, the data's gone. Your users will not be happy. + +3. You don't know if the email is unique. You are going to use the email as the userid, but a bunch of entries could be created with the same email. + +4. You stored the plain text password, which is very insecure. + +5. Only one user can be logged on at a time. + +Well ... we'll fix all of that, over time. + +### **Keeping Your Code Organized: Creating a Controller** + +You are going to have to create a couple more post routes. Also, you are going to have to add a lot of logic, to solve problems 1 through 5 above. You don't want all of that in app.js. So, create a directory called controllers. Within it, create a file called userController.js. Within that, create a function called register. The register() function takes a req and a res, and the body is just as above. You can move any `import` for shared helpers (such as a memory store module) into that file using a relative path. You should also `import` from `http-status-codes`, and instead of using 201, you use `StatusCodes.CREATED`. Then, export `register` from this module (for example `export async function register` or `export { register, ... }`). + +### **On Naming** + +In the general case, you can name modules and functions as you choose. However, we are providing tests for what you develop, and so you need to use the names specified below, so that the tests work: + +``` +/controllers/userController.js with functions logon, register, and logoff +/controllers/taskController.js with functions index, create, show, update, and deleteTask. +``` + +The show function returns a single task, and the index function returns all the tasks for the logged on user (or 404 if there aren't any.) + +### **Back to the Coding*** + +Change the code for the route as follows: + +```js +import { register } from "./controllers/userController.js"; +app.post("/api/user/register", register); +``` + +Test again with Postman to make sure it works. + +### **More on Staying Organized: Creating a Router** + +You are going to create several more user post routes, one for logon, and one for logoff. You could have app.post() statements in app.js for each. But as your application gets more complex, you don't want all that stuff in app.js. So, you create a router. Create a folder called routes. Within that, create a file called userRoutes.js. It should read as follows: + +```js +import express from "express"; +import { register } from "../controllers/userController.js"; + +const router = express.Router(); + +router.route("/register").post(register); + +export default router; +``` + +Then, change app.js to take out the app.post(). Instead, put this: + +```js +import userRouter from "./routes/userRoutes.js"; +app.use("/api/users", userRouter); +``` + +The user router is called for the routes that start with "/api/users". You don't include that part of the URL path when you create the router itself. + +All of the data sent or received by this app is JSON. You are creating a back end that just does JSON REST requests. So, you really shouldn't do res.send("everything worked."). You should always do this instead: + +```js +res.json({message: "everything worked."}); +``` + +At this time, change the res.send() calls you have in your app and middleware to res.json() calls. Remember that res.json() calls must return an object. If this is only a message, then for the sake of consistency, start that object with a `message` attribute. + +### **The Other User Routes** + +Here's a spec. + +1. You need to have an `/api/users/logon` POST route. That one would get a JSON body with an email and a password. The controller function has to do a find() on the `global.users` array for an entry with a matching email. If it finds one, it checks to see if the password matches. If it does, it returns a status code of OK, and a JSON body with the user name and email. The user name is convenient for the front end, because it can show who is logged on. The email may or may not be used by the front end, but you can return it. The controller function for the route would also set the value of `global.user_id` to be the entry in the `global.users` array that it finds. (You don't make a copy, you just set the reference.) If the email is not found, or if the password doesn't match, the controller returns an UNAUTHORIZED status code, with a message that says Authentication Failed. + +2. You need to have an `/api/users/logoff` POST route. That one would just set the `global.user_id` to null and return a status code of OK. You could do `res.sendStatus()`, because you don't need to send a body. + +3. You add the handler functions to the userController, and you add the routes to the userRoutes.js router, doing the necessary exports and requires. + +4. You test with Postman to make sure all of this works. + +5. Run the TDD test! You type `npm run tdd assignment3a` . + + +For the rest of this assignment, you'll set your app aside for a moment, and learn some debugging skills. + +--- + +## **Task 2: Debugging Middleware** + +### ***Introduction to the Scenario** + +You're volunteering for a local dog rescue, **The Good Boys and Girls Club**, to help them upgrade their adoption site. + +They’ve already built the main API routes, but their middleware is a mix of broken and missing. Your job is to clean things up and ensure the app behaves, just like all their dogs! + +The site serves adorable images of adoptable dogs, accepts applications from potential adopters, and includes a test route for simulating server errors. It just needs your help to become a robust, production-ready app using Express middleware the right way. + +You'll be implementing middleware that handles things like: + +* Logging and request tracking +* Request validation and parsing +* Serving dog images as static files +* Gracefully handling unexpected errors +The dogs are counting on you. + +### Setup + +1. The `week-3-middleware` folder is already provided in your repository. This folder contains the skeleton code for the dog rescue application. + +2. To run the provided framework enter ```npm run week3``` in terminal. You do this to start server before you begin testing with Postman. + +3. To run the tests (Vitest), enter ```npm run tdd assignment3b``` in terminal. Your task is to modify the existing files in the `week-3-middleware folder` to make the tests pass. + +### **Advanced Middleware Implementation** + +The dog rescue team wants to add more robust middleware to their application. Implement these additional features: + + +**Request Validation:** +- Add middleware that validates the `Content-Type` header for POST requests +- If a POST request doesn't have `application/json` content type, return a 400 error with a helpful message +- Include the request ID in the error response + +**Error Handling Middleware:** +- Create custom error classes that extend `Error` with status code properties +- Add middleware to catch different error types and return appropriate HTTP status codes: + - `ValidationError` → 400 Bad Request + - `NotFoundError` → 404 Not Found + - `UnauthorizedError` → 401 Unauthorized + - Default errors → 500 Internal Server Error +- Log errors with different severity levels based on status code (see Task 4 for details) + + +**Testing Your Implementation:** +- Test with Postman to ensure all new middleware works correctly +- Test that invalid content types return proper error responses +- Error responses include the correct status code, message, and requestId +- Unmatched routes return a 404 JSON response + + +4. In **Postman**, set up the following routes. They should all be in one collection called "dogs": + + * `GET {{host}}/dogs` + * `GET {{host}}/images/dachshund.png` + * `GET {{host}}/error` + * `POST {{host}}/adopt` + * Body: + + ```json + { + "name": "your name", + "address": "123 Sesame Street", + "email": "yourname@codethedream.org", + "dogName": "Luna" + } + ``` + Here `{{host}}` is a Postman environment variable you should configure. It should be set to `http://localhost:3000`. You'll do manual testing with Postman. + +5. Get coding! + +### Deliverables + +Your work will involve editing `app.js` to add the expected middleware. You will also need to modify `routes/dogs.js` to throw custom errors (`ValidationError` and `NotFoundError`) instead of returning error responses directly. + +**Important:** Pay attention to the **order** of your middleware! As you learned in Lesson 3, middleware executes in the order it's defined. Place each middleware in the correct position in the chain. + +**Recommended Middleware Order:** +1. Request ID middleware (adds `req.requestId`) +2. Logging middleware (logs requests with requestId) +3. Security headers middleware (sets security headers) +4. Body parsing middleware (`express.json()` with size limit) +5. Content-Type validation middleware (for POST requests) +6. Routes (your route handlers) +7. Error handling middleware (catches thrown errors) +8. 404 handler (catches unmatched routes) + +1. **Built-In Middleware** + + * The `POST /adopt` endpoint doesn’t seem to be processing requests as expected. This route expects a `name`, `email`, and `dogName`, but the controller keeps erroring. Implement the appropriate middleware to parse JSON requests on this endpoint. + * The images for adoptable dogs are not being served on `GET /images/**` as expected. Implement the appropriate middleware to serve the images of adoptable dogs from the `public/images/..` directory on this endpoint. + +2. **Custom Middleware** + + * The following middleware should be chained and applied globally to all routes: + * We would like to add a unique request ID to all incoming requests for debugging purposes. Using the `uuid` library to generate the unique value, write a custom middleware that: + * Adds a `requestId` property to `req` object (e.g., `req.requestId`) for all requests in the application + * Injects this value as an `X-Request-Id` header in the response headers (note: header name is case-insensitive, but use `X-Request-Id`) + * This middleware should run first, before any other middleware that might need the requestId + + **💡 Hint: Using the `uuid` Library** + - First, install the package: `npm install uuid` + - Import it at the top of your `app.js`: `import { v4 as uuidv4 } from "uuid";` + - Generate a unique ID: `const requestId = uuidv4();` (this creates a string like "9b1deb4d-3b7d-4bad-9bdd-2b0d7b3dcb6d") + - Example middleware structure: + ```js + app.use((req, res, next) => { + req.requestId = uuidv4(); + res.setHeader('X-Request-Id', req.requestId); + next(); + }); + ``` + + * We would like to output logs on all requests. These logs should contain the timestamp of the request, the method, path, and request ID. They should be formatted as: + + ```js + `[${timestamp}]: ${method} ${path} (${requestID})` + ``` + + **💡 Hint: Logging Format Requirements** + - Use `console.log()` to output these logs + - The timestamp can be formatted as an ISO string: `new Date().toISOString()` (e.g., "2024-01-15T10:30:45.123Z") + - The method should be `req.method` (e.g., "GET", "POST") + - The path should be `req.path` (e.g., "/dogs", "/adopt") + - The requestID should come from `req.requestId` set by the request ID middleware above + - Example implementation: + ```js + app.use((req, res, next) => { + const timestamp = new Date().toISOString(); + console.log(`[${timestamp}]: ${req.method} ${req.path} (${req.requestId})`); + next(); + }); + ``` + - **Important:** Make sure this middleware runs AFTER the request ID middleware, so that `req.requestId` is already set + +3. **Custom Error Handling** + +* Catch any uncaught errors and respond with a `500 Internal Server Error` error status and a JSON response body with the `requestId` (note: lowercase 'd') and an error message set to "Internal Server Error" +* The error response should be a JSON object: `{ error: "Internal Server Error", requestId: "..." }` +* You can test this middleware with the `/error` endpoint + +**💡 Hint: Basic Error Handling Middleware** + +Error handling middleware must have 4 parameters: `(err, req, res, next)`. Express recognizes it as an error handler because of the 4 parameters. + +```js +app.use((err, req, res, next) => { + res.status(500).json({ + error: "Internal Server Error", + requestId: req.requestId + }); +}); +``` + +**Note:** The error handling middleware should be placed after all routes but before the 404 handler, as error handlers must be the last middleware (except for 404 handlers). + +## **Task 3: Enhanced Middleware Features** + +The dog rescue team wants to add more robust middleware to their application. Implement these additional features: + +### **Request Size Limiting** +- Add middleware to limit request body size to prevent large requests from crashing the server +- Use `express.json({ limit: '1mb' })` for JSON request bodies +- This middleware should come before your routes but after security headers + +**💡 Hint: Request Size Limiting** +- The `limit` option in `express.json()` prevents the server from processing request bodies larger than the specified size +- If a request exceeds the limit, Express will automatically return a 413 (Payload Too Large) error +- The limit can be specified as a string like `'1mb'`, `'500kb'`, or `'10mb'` +- Example: `app.use(express.json({ limit: '1mb' }));` +- This helps protect your server from denial-of-service attacks where attackers send extremely large request bodies + +### **Content-Type Validation** +- Add middleware that validates the `Content-Type` header for POST requests +- If a POST request doesn't have `application/json` content type, return a 400 error with a helpful message +- The error message should match the pattern: `Content-Type must be application/json` +- Include the request ID in the error response (as `requestId` in the JSON response body) +- The error response should be: `{ error: "Content-Type must be application/json", requestId: "..." }` +- **Important:** This validation should only apply to POST requests - GET requests should not be validated +- This middleware should run after body parsing middleware but before routes + +**💡 Hint: Content-Type Validation Middleware** +- Check if the request method is POST: `req.method === 'POST'` +- Get the Content-Type header: `req.get('Content-Type')` or `req.headers['content-type']` +- Check if it equals `'application/json'` (case-insensitive comparison is recommended) +- If validation fails, send a 400 response with the error message and requestId +- If validation passes (or it's not a POST request), call `next()` to continue +- Example structure: + ```js + app.use((req, res, next) => { + if (req.method === 'POST') { + const contentType = req.get('Content-Type'); + if (!contentType || !contentType.includes('application/json')) { + return res.status(400).json({ + error: 'Content-Type must be application/json', + requestId: req.requestId + }); + } + } + next(); + }); + ``` + +### **404 Handler** +- Add a proper 404 handler that runs after all routes (this should be the very last middleware) +- It should return a JSON response with status 404 and include the request ID: + ```js + { + "error": "Route not found", + "requestId": "your-request-id" + } + ``` +- The requestId should be accessed from `req.requestId` (set by your request ID middleware) + +## **Task 4: Advanced Error Handling** + +Implement sophisticated error handling using custom error classes: + +### **Custom Error Classes** +- Create a new file called `errors.js` in the `week-3-middleware` folder +- Create a `ValidationError` class that extends `Error` with a status code property (400) +- Create a `NotFoundError` class for 404 errors +- Create an `UnauthorizedError` class for 401 errors +- Export all error classes from the `errors.js` file +- Import and use these error classes in your `app.js` and `routes/dogs.js` files + +**Note:** While `UnauthorizedError` is not tested in assignment3b, you should create it as you'll need it for future assignments that implement authentication. + +**💡 Hint: How to Create Custom Error Classes** + +In JavaScript, you can create custom error classes by extending the built-in `Error` class. Here's how: + +```js +// errors.js +class ValidationError extends Error { + constructor(message) { + super(message); // Call the parent Error constructor with the message + this.name = 'ValidationError'; // Set the error name (used for error identification) + this.statusCode = 400; // Add a custom property for the HTTP status code + } +} + +class NotFoundError extends Error { + constructor(message) { + super(message); + this.name = 'NotFoundError'; + this.statusCode = 404; + } +} + +class UnauthorizedError extends Error { + constructor(message) { + super(message); + this.name = 'UnauthorizedError'; + this.statusCode = 401; + } +} + +export { + ValidationError, + NotFoundError, + UnauthorizedError +}; +``` + +**Key Points:** +- `extends Error` makes your class inherit from JavaScript's built-in Error class +- `super(message)` calls the parent constructor to set the error message +- `this.name` should match the class name (this helps error handlers identify the error type) +- `this.statusCode` is a custom property you add for HTTP status codes +- Export the classes so they can be imported in other files + +**Usage Example:** +```js +// In routes/dogs.js +import { ValidationError, NotFoundError } from "../errors.js"; + +// Throw a ValidationError +if (!name || !email || !dogName) { + throw new ValidationError("Missing required fields"); +} + +// Throw a NotFoundError +if (!dog || dog.status !== "available") { + throw new NotFoundError("Dog not found or not available"); +} +``` + +**Important:** You will need to modify `routes/dogs.js` to throw these custom errors: + +1. **Add validation for required fields:** + - In the `/adopt` POST route, first check if required fields are present + - If `name`, `email`, or `dogName` are missing, throw a `ValidationError` with the exact message: `"Missing required fields"` + - This will result in a 400 Bad Request response + +2. **Add validation for dog existence and availability:** + - After checking required fields, check if the requested dog exists and is available + - Find the dog in the `dogData` array (imported from `../dogData.js`) by matching the `dogName` from the request body with the dog's `name` property in the array + - If the dog is not found in the array OR if the dog's `status` is not "available", throw a `NotFoundError` with a message that matches the pattern `/not found or not available/` + - Example messages that would match: `"Dog not found or not available"` or `"Dog not found or not available for adoption"` + - This will result in a 404 Not Found response + +3. **Error message requirements:** + - **The error messages must match exactly** - the test checks for specific patterns in the error messages + - For ValidationError: The message must match `/Missing required fields/` + - For NotFoundError: The message must match `/not found or not available/` + - If your error messages don't match these patterns, the tests will fail + +4. **Implementation details:** + - The error should be thrown (not returned with `res.status`), so that the error handling middleware can catch it + - Make sure to import the error classes at the top of `routes/dogs.js` + - **Note:** The success response (status 201 with message) should remain unchanged - only modify the error handling logic + +### **Error Handling Middleware** +- Add middleware to catch different error types and return appropriate HTTP status codes: + - `ValidationError` → 400 Bad Request + - `NotFoundError` → 404 Not Found + - `UnauthorizedError` → 401 Unauthorized + - Default errors → 500 Internal Server Error +- Log errors with different severity levels based on the error type: + - **4xx errors (400, 401, 404):** Use `console.warn()` to log these errors + - `ValidationError` (400) → Log with `console.warn("WARN: ValidationError", error.message)` or `console.warn("WARN: ValidationError " + error.message)` + - `UnauthorizedError` (401) → Log with `console.warn("WARN: UnauthorizedError", error.message)` or `console.warn("WARN: UnauthorizedError " + error.message)` + - `NotFoundError` (404) → Log with `console.warn("WARN: NotFoundError", error.message)` or `console.warn("WARN: NotFoundError " + error.message)` + - **5xx errors (500):** Use `console.error()` to log these errors + - Default errors (500) → Log with `console.error("ERROR: Error", error.message)` or `console.error("ERROR: Error " + error.message)` +- **Important:** The logged message must contain "WARN:" for 4xx errors and "ERROR:" for 5xx errors (the test uses `stringMatching` to check if the logged string contains these patterns) +- Ensure all error responses include the request ID for debugging +- All error responses should be JSON objects with `error` and `requestId` properties + +**💡 Hint: Error Handling Middleware Structure** + +Error handling middleware has 4 parameters: `(err, req, res, next)`. Express recognizes it as an error handler because it has 4 parameters. + +```js +app.use((err, req, res, next) => { + // Determine the status code from the error + const statusCode = err.statusCode || 500; + + // Log based on error type + if (statusCode >= 400 && statusCode < 500) { + // 4xx errors: client errors (use console.warn) + // This includes ValidationError (400), UnauthorizedError (401), NotFoundError (404) + console.warn(`WARN: ${err.name}`, err.message); + } else { + // 5xx errors: server errors (use console.error) + console.error(`ERROR: Error`, err.message); + } + + // Send error response + res.status(statusCode).json({ + error: err.message || 'Internal Server Error', + requestId: req.requestId + }); +}); +``` + +**Key Points:** +- Check `err.name` or `err.statusCode` to identify the error type +- Use `err.statusCode` if available, otherwise default to 500 +- For 4xx errors (400-499), use `console.warn()` with "WARN:" prefix +- For 5xx errors (500+), use `console.error()` with "ERROR:" prefix +- Always include `requestId` from `req.requestId` in the error response +- The error response should have `error` and `requestId` properties + +### **Security Headers** +- Add middleware that sets basic security headers: + - `X-Content-Type-Options: nosniff` + - `X-Frame-Options: DENY` + - `X-XSS-Protection: 1; mode=block` +- **Important:** This middleware should run for all responses, so place it early in your middleware chain (after request ID and logging, but before body parsing) +- These headers help protect against common web vulnerabilities + +## **Task 5: Testing Your Implementation** + +Test all your new middleware features: + +- **Basic Functionality:** Ensure all existing routes still work +- **Content-Type Validation:** Test that invalid content types return proper error responses +- **Error Handling:** Test different error types return appropriate status codes +- **Security Headers:** Check that security headers are present in all responses +- **404 Handling:** Test that unmatched routes return proper 404 responses + +### Checking Your Work + +You start the server for this exercise with `npm run week3`. You stop it with a Ctrl-C. You run `npm run tdd assignment3b` to run the Vitest tests for this exercise. Also use Postman to test. Confirm the responses in Postman and the logs in your server terminal match the expectations in the deliverables. + +### **💡 Tips for Passing the TDD Tests (Part B)** + +If your Postman tests are working but the Vitest tests are failing, here's a troubleshooting approach: + +1. **Examine the Test File:** + - Open `tdd/assignment3b.test.js` in your repository + - Read through the test cases to understand exactly what the tests expect + - Pay attention to the exact error messages, status codes, and response body formats + +2. **Check Test Error Messages:** + - When a test fails, read the error message carefully + - The test error messages will tell you exactly what the test expects: + - The exact status code + - The exact error message text in the response body + - The exact format of the response (JSON structure) + - The exact log message format + - Compare your implementation with these expectations and adjust accordingly + +3. **Review the Main Instructions:** + - Make sure you've implemented all requirements from **Task 4: Advanced Error Handling**: + - Custom error classes (ValidationError, NotFoundError, UnauthorizedError) + - Error handling middleware with proper logging (console.warn for 4xx, console.error for 5xx with "WARN:" and "ERROR:" prefixes) + - Validation for dog existence in `routes/dogs.js` (see Task 4, section 2) + - Verify that error messages match the exact patterns specified in Task 4 + +## Video Submission + +Record a short video (3–5 minutes) on YouTube (unlisted), Loom, or similar platform. Share the link in your submission form. + +**Video Content**: Answer 3 questions from Lesson 3: + +1. **What is the architecture of an Express application and how do its components work together?** + - Explain the main architectural components: app instance, middleware, route handlers, and error handlers + - Discuss the order of middleware execution and why it matters + - Demonstrate the difference between middleware and route handlers + +2. **How do you handle HTTP requests and responses in Express?** + - Explain the structure of HTTP requests (method, path, headers, body) + - Show how to access request data (req.method, req.path, req.body, req.query) + - Demonstrate response methods (res.json(), res.send(), res.status()) + +3. **What is REST and how does it relate to Express applications?** + - Explain REST principles and HTTP methods + - Show how to design RESTful API endpoints + - Discuss HTTP status codes and proper API responses + +**Video Requirements**: +- Keep it concise (3-5 minutes) +- Use screen sharing to show code examples +- Speak clearly and explain concepts thoroughly +- Include the video link in your assignment submission + +## **Submit Your Assignment on GitHub** + +📌 **Follow these steps to submit your work:** + +#### **1️⃣ Add, Commit, and Push Your Changes** + +- Within your node-homework folder, do a git add and a git commit for the files you have created, so that they are added to the `assignment3` branch. +- Push that branch to GitHub. + +#### **2️⃣ Create a Pull Request** + +- Log on to your GitHub account. +- Open your `node-homework` repository. +- Select your `assignment3` branch. It should be one or several commits ahead of your main branch. +- Create a pull request. + +#### **3️⃣ Submit Your GitHub Link** + +- Your browser now has the link to your pull request. Copy that link. +- Paste the URL into the **assignment submission form**. +- **Don't forget to include your video link in the submission form!** + + diff --git a/lessons/03-express-middleware.md b/lessons/03-express-middleware.md new file mode 100644 index 0000000..4fd0c0a --- /dev/null +++ b/lessons/03-express-middleware.md @@ -0,0 +1,764 @@ +# **Lesson 3 — Express Application Concepts** + +## **Lesson Overview** + +**Learning objective**: Students will get some foundational knowledge of Internet protocols, REST APIs, and JSON. Students will learn more about what Express is and how it augments Node to make an easy and comprehensive framework for the development of web applications. Students will learn the basic elements of an Express application and the purpose of each. + +**Topics** + +1. Review: What is Express? +2. Internet Basics +3. HTTP +4. REST and JSON +5. What do Route Handlers Do In Express? +6. Middleware Functions, Route Handlers, and Error Handling +7. Parsing an HTTP Request +8. Debugging an Express Application + +### **Course defaults: ECMAScript modules and Vitest** + +In this course, Node projects are configured as **ESM** (ECMAScript modules), not CommonJS. Your `package.json` should include `"type": "module"`. Use `import` and `export` instead of `require()` and `module.exports`. For relative imports in Node ESM, include the file extension, for example `import { register } from "./controllers/userController.js"`. + +Automated tests for assignments use **Vitest** as the test runner (not Jest). Vitest APIs will feel familiar if you have used Jest—`describe`, `it`/`test`, `expect`, mocks, and async tests work in the same style. Run tests with the npm scripts provided in your homework repo (for example `npm run tdd` where that script invokes Vitest). + +## **3.1 Review: What is Express** + +As we have seen, all of the elements needed to create a web application are provided by Node, including network API access, the HTTP server, event handlers, streams, etc. Express assembles these in an easy to use framework, where the flow of control is easily understood and where each of the elements is compact and easily created. Express is very widely used, and as a result, it has a comprehensive ecosystem of additional plug-in packages that facilitate data exchange, HTTP request parsing, and the construction of HTTP responses for the applications you create. But, to use it effectively, you need to understand the protocols and data flows involved, so this lesson describes them. + +## **3.2 Internet Basics** + +All Internet traffic is based on layers of protocols. The Internet runs on IP: Internet Protocol. When data is sent over the Internet, it is broken up into packets. Each packet has a source address, a destination address, a protocol and a port. The addresses are four part numbers like 9.28.147.56. The protocol is also a number, indicating what kind of packet it is, and the port is also a number, which endpoints use to figure out which process on a machine should get the packet. A network of routers figures out where the destination machine is and forwards the packet. + +For REST, you will use a protocol on top of IP called TCP, which stands for Transmission Control Protocol. TCP has reliable connections, where "reliable" means that each TCP endpoint sends acknowledgements when a packet arrives, and if the acknowledgement is slow in arriving, the source machine sends the packet again. Retries continue until the acknowledgement arrives or a timeout occurs. TCP connections have a server, which listens for connection requests, and a client, which initiates them. Your browser is a client. Servers and clients communicate over TCP using a programming interface called sockets, but sockets are just a programming interface of the operating system. Servers typically have a DNS name, like www.widgets.com. DNS stands for Domain Name System, and a network of domain name servers keep track of the names so that each endpoint can look them up. TCP connections can be augmented with SSL, which stands for Secure Sockets Layer. There are two advantages to SSL. First, all the data sent by either end of the connection is encrypted so that no one can listen in. Second, when the SSL connection is established, the server proves, by means of a cryptographic exchange, that it really is www.widgets.com, and not some impostor. + +Remember all of this. It might come up during trivia night at your local bar. + +## **3.3 HTTP** + +REST requests flow over a protocol called HTTP, which stands for Hypertext Transfer Protocol. HTTP over an SSL connection is called HTTPS. Each HTTP request uses one of a small number of methods. + +- GET +- POST +- PUT +- PATCH +- DELETE +- HEAD +- OPTIONS +- CONNECT +- TRACE + +For REST requests, GET, POST, PUT, PATCH, and DELETE are used. Each HTTP request packet has various parts, most of which are optional: + +- A method. This is always present. + +- A path, like "/info/dogs", which comes from the URL. This is always present. + +- Query parameters, like "name="Spot". If present, these come from the URL, for example as in "http://info/dogs?name=Spot&owner=Frank. These are often used in REST requests. + +- URL or path parameters. If present, these are additional parameters parsed from the URL. + +- Headers. These are key-value pairs. For REST requests, the "Content-Type" header is always used, and it typicallyoften has the value "application/json". There are many other headers. + +- A body. POST, PUT and PATCH requests often have a body. Responses for each of the methods also often have a body. For REST, the body is usually JSON. By convention, POST operations are used to create some data on the back end, PATCH to update that data, and PUT to replace that data. Never use GET requests to change data! + +For every HTTP request, there is exactly one HTTP response (although the body of the response, if it is long, might be broken up into chunks. Each HTTP response packet also has components: + +- Headers +- A result code +- Often, a body. + +Pay attention to this part. You will use all the REST operations. You will need to specify the path, the query parameters, a header, JSON in the body, and even a cookie, to complete your final project. + +### **Stuff the Browser Keeps Track Of** + +Not all HTTP requests originate at a browser, but for those that do, the browser keeps track of information associated with the request. The browser keeps track of the origin for a request. This is the address and port for the URL that is the target of the request. When a browser application makes a REST request, that may go to the origin from which the application was loaded, or it might be a fetch() call to a different origin. If it goes to a different origin, that's a cross origin request. + +The browser also keeps track of cookies, which are key-value pairs. These are set because a Set-Cookie header was sent in a server response, and they store data, typically small amounts, but certainly less than 4k. They also have various flags. Browsers have policies for which Set-Cookie headers will be honored, and silently discard the rest. Cookies can be sent on subsequent requests from the browser application, depending on the request and also on the browser policies, until such time as the cookie expires. Cookie content is available to the JavaScript in the browser application, unless it is an HttpOnly cookie. A server sets an HttpOnly cookie to store information about the client, such as whether the user has logged on and who that user is. Cookies are not usually used unless the requesting application is running in a browser. + +## **3.4 REST and JSON** + +REST stands for Representational State Transfer, which is a pretty opaque name for a standard. What it means is that HTTP requests and responses are exchanged, and management of the state of the conversation and the security governing the exchange is not a part of the REST protocol itself. + +JSON stands for JavaScript Object Notation. You need to know JSON. Here is a [video introduction to JSON](https://www.youtube.com/watch?v=iiADhChRriM). Below is a summary. + +The types in JSON are: + +- Number, either an integer or a decimal. +- String, always in Unicode +- Boolean, either true or false +- Array, which is an ordered list of any of these datatypes. +- Object, a collection of name-value pairs. The names are always strings. The values can be any of these datatypes. +- null. + +You can put objects in objects, objects in arrays, arrays in objects, and so on, nesting as much as you like, so as to make the document as complicated as necessary. A JSON document must either be an object or an array. Here is an example from Wikipedia: + +```JSON +{ + "first_name": "John", + "last_name": "Smith", + "is_alive": true, + "age": 27, + "address": { + "street_address": "21 2nd Street", + "city": "New York", + "state": "NY", + "postal_code": "10021-3100" + }, + "phone_numbers": [ + { + "type": "home", + "number": "212 555-1234" + }, + { + "type": "office", + "number": "646 555-4567" + } + ], + "children": [ + "Catherine", + "Thomas", + "Trevor" + ], + "spouse": null +} +``` + +A JSON document is not a JavaScript object. The keys in a JSON object are always strings in double quotes. The string values in a JSON object are always specified in double quotes. In JavaScript, a JSON document is just a string. The following JavaScript functions are often used with JSON: + +```js +const anObject = JSON.parse(aJSONString); // convert from a JSON string to a JavaScript object. +const aJSONString = JSON.stringify(anObject); // convert from a JavaScript object to a JSON string. +``` + +Of course, not all JavaScript objects can be converted to JSON. If the object contains functions, for example, they are omitted from the resulting JSON string, and this also happens with other JavaScript types. + +Binary objects like JPEGs are never sent in JSON. You can still do a REST request for these datatypes, but you use a different content type. + +JSON objects can be parsed or created in any modern computer programming language: Python, Java, Rust, C++, etc.. In some NoSQL databases like MongoDB, every entry in the database is basically a JSON object. + +## **3.5 What do Route Handlers Do In Express?** + +For each HTTP request sent to a server, there must be exactly one response. If no response is sent, a user might be waiting at the browser for a timeout. If several responses are sent for one request, Express reports an error instead of sending the second one. A route is specified by a **method** (GET, POST, PUT, PATCH, DELETE, and several others) and a **path**, a part of the URL after the host and port, but before the query parameters, something like `/info/books`. + +Associated with each route in Express is a route handler, the function that Express calls to interpret the request and send back the response. Route handlers may retrieve data and send it back to the caller. Or, they may store, modify, or delete data, and report the success or failure to the caller. Or, they may manage the session state of the caller, as would happen, for example, with a logon. The data accessed by route handlers may be in a database, or it may be accessed via some network request. When sending the response, a route handler might send plain text, HTML, or JSON, or any number of other content types. A route handler must either send a response or call the error handler to send a response. Otherwise the request from the caller will wait until timeout. + +Route handlers and middleware functions frequently do asynchronous operations, often for database access. While the async request is being processed, other requests may come to the server, and they are dispatched as usual. Route handlers and middleware may be declared as async, so that the async/await style of programming can be used. These functions don't return a value of interest -- the interesting stuff is in the response, not the return value. + +### **Understanding Express Request Processing** + +Before diving into middleware details, it's important to understand how Express processes requests. Every request goes through a **middleware chain** - a series of functions that execute in a specific order: + +``` +1. Request arrives +2. Middleware functions execute in order (if they match the request) +3. Route handler executes (if route matches) +4. Response is sent +``` + +**Key Concept:** Middleware functions are like checkpoints that every request must pass through. They can: +- **Process the request** (log it, parse data, check authentication) +- **Modify the request** (add data to `req` object) +- **Send a response** (end the chain early) +- **Pass control to the next middleware** (call `next()`) + +This is why **order matters** - you can't parse JSON data after you try to use it! + +## **3.6 Middleware Functions, Route Handlers, and Error Handling** + +Let's sum up common characteristics of middleware functions and response handlers. Let's also explain how errors are handled. + +### **Understanding the Middleware Chain** + +Express processes requests through a **middleware chain** - a series of functions that execute in the order they are defined. Understanding this execution order is crucial for building effective Express applications. + +#### **Middleware Chain Execution Order** + +``` +Request → Middleware 1 → Middleware 2 → Middleware 3 → Route Handler → Response +``` + +**Key Points:** +- Middleware functions execute **in the order they are defined** with `app.use()` +- Each middleware can either: + - Process the request and call `next()` to continue to the next middleware + - Send a response and end the chain + - Pass an error to the error handler +- **Order matters!** A middleware that parses JSON must come before routes that need `req.body` +- Route handlers are just the final middleware in the chain + + +### **Common Characteristics of Middleware and Route Handlers** + +1. They are each called with the parameters req and res, or possibly req, res, and next. They may be declared as async functions. + +2. Once they are called, these functions do processing based on the information in the req object: method, path, path parameters, query parameters, headers, cookies, the body. Every request has a method and path, but the other request attributes may or may not be present. + +3. These functions must do one of the following, or the request times out: + +- Send a response. +- Call next(). +- Throw an error. + +Even route handlers sometimes call `next(error)` to pass the error to the error handler. Middleware functions often call next() withouut parameters, to call the next middleware in the chain or the route handler for the request, but they also might call `next(error)` in some cases. + +4. If `next(error)` is called or an error is thrown, the error handler is called and passed the error. An error might be thrown from the code of the middleware function or route handler. Or it might be thrown by one of the function calls that the middleware function or route handler makes. In the latter case, if it is a known type of error, the middleware function or route handler may catch the error and send an appropriate response to the requester. But if it is not an error of known type, it is a 500: an internal server error. Route handlers and middleware functions don't need to catch unknown error types, and if they do, they can just throw them again. + +The Express 5 error handler catches all the errors thrown by middleware functions and route handlers, and also receives all errors that are reported to it using `next(error)`. This happens even if the error is thrown by an asynchronous function call. + +**However, please note:** Middleware functions and route handlers sometimes call functions that have callbacks. They may send responses or call next() from within the callback. That works fine. But they must **never** throw an error from within a callback. That would crash the server. They must call `next(error)` instead. + +### **Common Middleware Order Pattern** + +Here's a typical Express application structure showing the correct order of middleware: + +```js +import express from "express"; +// import apiRoutes from "./routes/api.js"; +// import adminRoutes from "./routes/admin.js"; +const app = express(); + +// 1. Logging (first - logs all requests) +app.use((req, res, next) => { + console.log(`${new Date().toISOString()}: ${req.method} ${req.path}`); + next(); +}); + +// 2. Body parsing (before routes that need req.body) +app.use(express.json()); // Parse JSON bodies +app.use(express.urlencoded({ extended: true })); // Parse form data + +// 3. Static files (before routes) +app.use(express.static("public")); + +// 4. Routes +app.use("/api", apiRoutes); +app.use("/admin", adminRoutes); + +// 5. 404 handler (after all routes) +app.use((req, res) => { + res.status(404).json({ message: "Route not found" }); +}); + +// 6. Error handler (last - catches all errors) +app.use((err, req, res, next) => { + console.error(err.stack); + res.status(500).json({ message: "Internal server error" }); +}); +``` + +**Key Points:** +- **Logging first** - You want to see all requests +- **Body parsing early** - Routes need `req.body` to be parsed +- **Static files before routes** - Serves files without hitting route handlers +- **Routes in the middle** - Your application logic +- **404 handler after routes** - Catches unmatched routes +- **Error handler last** - Catches any errors from the chain + +## **3.7 Parsing an HTTP Request** + +One very common piece of middleware is the following: + +```js +app.use(express.json()) +``` + +This middleware parses the body of a request that has content-type "application/json". The resulting object is stored in req.body. There are other body parsers to be used in other circumstances, for example to catch data that is posted from an HTML form. + +### **The req and res Objects** + +You can access the following elements of the req: + +req.method +req.path +req.params HTML path parameters, if any. When you configure a route with a route handler, you can tell Express where these are in the URL. +req.query query parameters of the request, if any +req.body The body of the request, if any +req.host The host that this Express app is running on + +There are many more. + +The req.get(headerName) function returns the value of a header associated with the request, if that header is present. +`req.cookies[cookiename]` returns the cookie of that name associated with request, if one is present. + +The res object has the following methods: + +- res.status() : This sets the HTTP status code +- res.cookie() : Causes a Set-Cookie header to be attached to the response +- res.setHeader() : Sets a header in the response +- res.json() : When passed a JavaScript object, this method converts the object to JSON and sends it back to the originator of the request +- res.send() : This sends plain text data, or perhaps HTML + +## **3.8 Built-in vs. Custom Middleware** + +Understanding the difference between built-in and custom middleware is crucial for building effective Express applications. The following sections give several examples for illustration, but you don't need to put them in your code at this time. + +### **Built-in Middleware** + +Express provides several built-in middleware functions that handle common web application tasks: + +#### **Body Parsing Middleware** +```js +// Parse JSON request bodies +app.use(express.json({ limit: "1mb" })); + +// Parse URL-encoded form data +app.use(express.urlencoded({ extended: true })); + + +``` + +#### **Static File Serving** + +```js +// Serve static files from 'public' directory +app.use(express.static("public")); + +// Serve static files with custom path prefix +app.use("/static", express.static("public")); +``` + +#### **Third-party middleware** +These are created and maintained as separate npm packages. The following are examples -- not something you need to use as this time. + +```js +import cookieParser from "cookie-parser"; +app.use(cookieParser()); + +import compression from "compression"; +app.use(compression()); +``` + +### **Custom Middleware** + +Custom middleware functions are functions you write to handle specific application logic: + +#### **Request Modification Middleware** + + +```js +// Add custom properties to request object +app.use((req, res, next) => { + req.timestamp = new Date().toISOString(); + req.requestId = Math.random().toString(36).substr(2, 9); + next(); +}); + +// Add user information to request +app.use((req, res, next) => { + req.user = getCurrentUser(req); + next(); +}); +``` + +#### **Response Modification Middleware** +```js +// Add custom headers to all responses +app.use((req, res, next) => { + res.setHeader("X-Powered-By", "MyApp"); + res.setHeader("X-Request-ID", req.requestId); + next(); +}); + +``` + + +## **3.9 Request and Response Modification** + +Middleware functions can modify both incoming requests and outgoing responses to add functionality or transform data. + +### **Request Modification** + +Middleware can add properties to the `req` object that subsequent middleware and route handlers can use: + +```js +// Add timing information +app.use((req, res, next) => { + req.startTime = Date.now(); + next(); +}); + +// Add user context +app.use((req, res, next) => { + const token = req.headers.authorization; + if (token) { + req.user = decodeToken(token); + } + next(); +}); + +// Add request metadata +app.use((req, res, next) => { + req.metadata = { + userAgent: req.get("User-Agent"), + ip: req.ip, + timestamp: new Date().toISOString() + }; + next(); +}); +``` + +### **Response Modification** + +Middleware can modify responses before they're sent to the client: + +```js +// Add security headers +app.use((req, res, next) => { + res.setHeader("X-Content-Type-Options", "nosniff"); + res.setHeader("X-Frame-Options", "DENY"); + res.setHeader("X-XSS-Protection", "1; mode=block"); + next(); +}); + +// Add CORS headers +app.use((req, res, next) => { + res.setHeader("Access-Control-Allow-Origin", "*"); + res.setHeader("Access-Control-Allow-Methods", "GET, POST, PUT, DELETE"); + res.setHeader("Access-Control-Allow-Headers", "Content-Type, Authorization"); + next(); +}); + +``` + +## **3.10 Comprehensive Error Handling** + +Error handling in Express involves catching errors and sending appropriate responses to clients. + +### **HTTP Status Codes** + +HTTP status codes are three-digit numbers that indicate the result of an HTTP request. They're grouped into five categories: + +#### **1xx Informational (100-199)** +Rarely used in web applications. Indicates the request was received and processing continues. + +#### **2xx Success (200-299)** +The request was successful: + +```js +// 200 OK - Request successful +res.status(200).json({ message: "Success", data: result }); + +// 201 Created - Resource created successfully +res.status(201).json({ message: "User created", user: newUser }); + +// 204 No Content - Success but no content to return +res.status(204).send(); +``` + +#### **3xx Redirection (300-399)** +The request needs further action: + +```js +// 301 Moved Permanently +res.status(301).redirect("/new-path"); + +// 302 Found (temporary redirect) +res.status(302).redirect("/temporary-path"); +``` + +#### **4xx Client Errors (400-499)** +The client made an error: + +```js +// 400 Bad Request - Invalid request data +app.use((req, res, next) => { + if (!req.body || Object.keys(req.body).length === 0) { + return res.status(400).json({ + error: "Bad Request", + message: "Request body is required" + }); + } + next(); +}); + +// 401 Unauthorized - Authentication required +app.use('/api', (req, res, next) => { + if (!req.headers.authorization) { + return res.status(401).json({ + error: "Unauthorized", + message: "Authentication required" + }); + } + next(); +}); + +// 403 Forbidden - Authenticated but not authorized +app.use('/admin', (req, res, next) => { + if (req.user && req.user.role !== "admin") { + return res.status(403).json({ + error: "Forbidden", + message: "Admin access required" + }); + } + next(); +}); + +// 404 Not Found - Resource doesn't exist +app.use((req, res) => { + res.status(404).json({ + error: "Not Found", + message: `Route ${req.method} ${req.path} not found` + }); +}); + +// 422 Unprocessable Entity - Valid request but invalid data +app.use((req, res, next) => { + if (req.body.email && !isValidEmail(req.body.email)) { + return res.status(422).json({ + error: "Unprocessable Entity", + message: "Invalid email format" + }); + } + next(); +}); +``` + +#### **5xx Server Errors (500-599)** +The server encountered an error: + +```js +// 500 Internal Server Error - Generic server error +app.use((err, req, res, next) => { + console.error("Server Error:", err); + res.status(500).json({ + error: "Internal Server Error", + message: "Something went wrong on our end" + }); +}); + +// 502 Bad Gateway - Server acting as gateway received invalid response +// 503 Service Unavailable - Server temporarily unavailable +// 504 Gateway Timeout - Server acting as gateway timed out +``` + +### **Common Status Code Patterns** + +#### **REST API Status Codes** +```js +// GET /users - 200 OK (with data) +app.get("/users", (req, res) => { + res.status(200).json({ users: allUsers }); +}); + +// POST /users - 201 Created +app.post("/users", (req, res) => { + const newUser = createUser(req.body); + res.status(201).json({ user: newUser }); +}); + +// PUT /users/:id - 200 OK (updated) or 201 Created (new) +app.put("/users/:id", (req, res) => { + const user = updateUser(req.params.id, req.body); + res.status(200).json({ user }); +}); + +// DELETE /users/:id - 204 No Content +app.delete("/users/:id", (req, res) => { + deleteUser(req.params.id); + res.status(204).send(); +}); +``` + +#### **Error Status Codes** +```js +// Validation errors - 400 Bad Request +if (!req.body.name) { + return res.status(400).json({ + error: "Bad Request", + message: "Name is required" + }); +} + +// Authentication errors - 401 Unauthorized +if (!isValidToken(req.headers.authorization)) { + return res.status(401).json({ + error: "Unauthorized", + message: "Invalid or missing token" + }); +} + +// Authorization errors - 403 Forbidden +if (req.user.role !== "admin") { + return res.status(403).json({ + error: "Forbidden", + message: "Insufficient permissions" + }); +} + +// Not found errors - 404 Not Found +const user = findUser(req.params.id); +if (!user) { + return res.status(404).json({ + error: "Not Found", + message: "User not found" + }); +} +``` + +### **Error Response Structure** + +Create consistent error responses across your application: + +```js + +// Error response utility +function createErrorResponse(statusCode, message) { + return { + message, + statusCode, + timestamp: new Date().toISOString() + }; +} + +``` + +### **Error Handling Middleware** + +```js +// Global error handler (place at the end of all routes) +app.use((err, req, res, next) => { + console.error("Error occurred:", err.message); + + const statusCode = err.statusCode || 500; + const message = err.message || "Internal Server Error"; + + res.status(statusCode).json({ + error: true, + message, + statusCode, + timestamp: new Date().toISOString() + }); +}); + + // Determine error type and response + app.use((err, req, res, next) => { + console.error("Error occurred:", { + message: err.message, + stack: err.stack, + url: req.url, + method: req.method, + timestamp: new Date().toISOString() + }); + + if (err.name === "ValidationError") { + return res.status(400).json(createErrorResponse(400, "Validation failed", err.details)); + } + + if (err.name === "UnauthorizedError") { + return res.status(401).json(createErrorResponse(401, "Authentication required")); + } + + if (err.name === 'CastError') { + return res.status(400).json(createErrorResponse(400, "Invalid ID format")); + } + + // Default to 500 error + res.status(500).json(createErrorResponse(500, "Internal server error")); +}); +``` + +### **Error Handling Best Practices** + +1. **Always Send a Response**: Never leave requests hanging +2. **Log Errors**: Include relevant context in error logs +3. **Use Appropriate Status Codes**: Follow HTTP status code conventions +4. **Provide Helpful Messages**: Give clients actionable error information +5. **Handle Async Errors**: Use try-catch or proper error handling for async operations + +## **3.11 Debugging an Express Application** + +There are various techniques to debug an Express application. + +First, you should try to produce the error condition. You do this one of two ways. You may send a browser request to the application, in which case the network tab of browser developer tools will show you exactly what was sent and received for each request. Or, you may use Postman to send requests, and Postman will show you exactly what was sent and received. + +Second, if you have a hunch what is causing the error, you put console.log() or console.info() or console.debug() statements. You would do this, in particular, if an error condition is happening in test or production that you can't duplicate. When the tester or user triggers the error, your log may tell you what is going on. You'll use this technique plenty in development too. + +Third, you can write debugging middleware that can identify and log the error condition. Your middleware function could report the content of the request, and perhaps some information about the response. In some cases, you could insert a middleware function that is to be called after the middleware you want to debug, to see if that is working correctly. + +Fourth, you could use the debugger built into VSCode. + +### **Using the VSCode Debugger** + +1. In your node-homework folder, there should be a `.vscode/launch.json` file. Open that file. If you do not have it, please go to the VS code navigation tools on the left hand side and click on "Run and Debug ". In the "Run and Debug" window click on the highlighted "create a launch.json file". When prompted to choose the debugger, please choose node.js. Your project should use `"type": "module"` in `package.json` so that `app.js` loads as ESM. + +2. Once you have that file opened, include the following configuration in the file: +```js +// .vscode/launch.json +{ + "version": "0.2.0", + "configurations": [ + { + "type": "node", + "request": "launch", + "name": "Launch Express App", + "runtimeExecutable": "nodemon", + "runtimeArgs": [ + "--inspect-brk", + "app.js" + ], + "console": "integratedTerminal", + "internalConsoleOptions": "neverOpen", + "restart": true, + "env": { + "NODE_ENV": "development", + "PORT": "3000" + } + } + ] +} +``` + +3. Next, add .vscode/ to your .gitignore. + +4. Edit a source code file, such as app.js. There are line numbers for each line in the file. If you click just to the left of the line number, a little red dot appears. This is a breakpoint. You can put them where ever you like in your code. Click again to remove the breakpoint. + +5. At the top of your VSCode window, there is a Run tab. Click on this and select "start debugging". + +Voilà. Your program runs to the first breakpoint. Your source file is displayed, and the program statement that is to run next is highlighted. On the left side of your VSCode window, you see the debugger screen. The Variables section shows all the variables that are active in the context. You can edit the values associated with each. You can add variable names to the Watch section, so you can see how they change as you step through your program. If you open the Call Stack section, you see the call stack. An entry in the call stack will say `paused` or `paused on breakpoint`. If you hover your mouse pointer over that, you see icons for continue, step over, step into, step out of, restart, and stop. If the next line is a function call, step over runs the entire function call, step into goes into the function, and step out of runs all instructions until the return from the current function has completed. If you do hit the continue button, it will run the program, and your Call Stack section will display a pause button that is usually ineffective. You should first set a breakpoint before you hit continue, or add a breakpoint as it runs and then send a Postman request that will cause the breakpoint to be reached. At the top of your VSCode window is a red square. You click on that to end the debug session. + +### **Sample Middleware for Debugging** + +The following middleware function might be helpful. + +```js +app.use((req, res, next) => { + res.on("finish", () => { +// Here you'd log information that might be helpful to know about the req and/or the res. + }); + next(); +}); +``` +The "finish" event fires after the response has been sent to the requester. You'd put this middleware function into the chain before any middleware function or request handler you are trying to debug. You have to register the res.on() finish event callback before the response is actually sent, or it won't catch anything. + +This middleware can log whatever is interesting in the req and res objects, with one exception. You can get the headers and result code from the response, but you can't get the body of the response. The body will already have been streamed to the network in response to the request. + +### **Check for Understanding** + +1. What are the parameters always passed to a route handler? What are they for? + +2. What must a route handler always do? + +3. How does a middleware function differ from a route handler? + +4. If you do an await in a route handler, who has to wait? + +5. You add a middleware function or router to the chain with an app.use() statement. You add a route handler to the chain with an app.get() or similar statement. Each of these has filter conditions. How do the filter conditions differ? + +### **Answers** + +1. A route handler always gets the req and res objects. The req object contains information from the request, perhaps with additional attributes added by middleware functions. The res object has methods including res.send() and res.json() that enable the route handler to respond to the request. + +2. A route hander must either respond to the request with res.send() or res.json(), or call the error handler with next(error), or throw the error. + +3. A middleware function may or may not respond to the request. Instead it may call next() to pass control to the next handler or middleware function in the chain. It must either respond to the request or call next, or perhaps throw an error. + +4. If you do an await in a route handler, the caller for the request has to wait. You might be waiting, for example, on a response from the database. You need the database response before you can send the HTTP response to the caller. On the other hand, other callers don't have to wait, unless they make a request that ends up at an await statement. + +5. An app.use() statement has an optional parameter, which is the path prefix. You might have a statement like app.use("/api", middlewareFunction), and then middlewareFunction would be called for every request with a path starting with "/api", no matter if it is a GET, a POST, or whatever. If the path prefix is omitted, the middleware function is called for every request. An app.get() statement, on the other hand, calls the corresponding route handler only if the path matches exactly and the method is a GET. In either case, you can pass additional middleware functions as parameters to be called in order, like: + +```js +app.use("/api", middleware1, middleware2, middleware3); +app.get("/info", middleware1, infoHandler); +``` + +(Examples in this lesson use ESM syntax; ensure your entry file and `package.json` are set up for `"type": "module"`.) \ No newline at end of file diff --git a/mentor-guidebook/sample-answers/assignment3/app.js b/mentor-guidebook/sample-answers/assignment3/app.js new file mode 100644 index 0000000..a6119c9 --- /dev/null +++ b/mentor-guidebook/sample-answers/assignment3/app.js @@ -0,0 +1,75 @@ +import express from "express"; +import userRoutes from "./routes/userRoutes.js"; +import errorHandler from "./middleware/error-handler.js"; +import notFound from "./middleware/not-found.js"; + +global.user_id = null; +global.users = []; +global.tasks = []; + +const app = express(); + +app.use((req, res, next) => { + console.log("Method:", req.method); + console.log("Path:", req.path); + console.log("Query:", req.query); + next(); +}); +const port = process.env.PORT || 3000; + +app.use(express.json()); + +app.use("/api/users", userRoutes); + +app.get("/health", (req, res) => { + res.json({ status: "ok" }); +}); + +app.use(notFound); +app.use(errorHandler); + +let server; +if (!process.env.VITEST) { + server = app.listen(port, () => + console.log(`Server is listening on port ${port}...`), + ); + + server.on("error", (err) => { + if (err.code === "EADDRINUSE") { + console.error(`Port ${port} is already in use.`); + } else { + console.error("Server error:", err); + } + process.exit(1); + }); + + let isShuttingDown = false; + async function shutdown(code = 0) { + if (isShuttingDown) return; + isShuttingDown = true; + console.log("Shutting down gracefully..."); + try { + await new Promise((resolve) => server.close(resolve)); + console.log("HTTP server closed."); + } catch (err) { + console.error("Error during shutdown:", err); + code = 1; + } finally { + console.log("Exiting process..."); + process.exit(code); + } + } + + process.on("SIGINT", () => shutdown(0)); + process.on("SIGTERM", () => shutdown(0)); + process.on("uncaughtException", (err) => { + console.error("Uncaught exception:", err); + shutdown(1); + }); + process.on("unhandledRejection", (reason) => { + console.error("Unhandled rejection:", reason); + shutdown(1); + }); +} + +export default app; diff --git a/mentor-guidebook/sample-answers/assignment3/controllers/userController.js b/mentor-guidebook/sample-answers/assignment3/controllers/userController.js new file mode 100644 index 0000000..ad563b1 --- /dev/null +++ b/mentor-guidebook/sample-answers/assignment3/controllers/userController.js @@ -0,0 +1,50 @@ +import { StatusCodes } from "http-status-codes"; + +export const register = async (req, res) => { + const existingUser = global.users.find( + (user) => user.email === req.body?.email, + ); + if (existingUser) { + return res.status(StatusCodes.BAD_REQUEST).json({ message: "User already exists" }); + } + + const newUser = { ...req.body }; + global.users.push(newUser); + + res.status(StatusCodes.CREATED).json({ + name: newUser.name, + email: newUser.email, + }); +}; + +export const logon = async (req, res) => { + const { email, password } = req.body; + + if (!email || !password) { + return res + .status(StatusCodes.BAD_REQUEST) + .json({ message: "Email and password are required" }); + } + + const user = global.users.find( + (u) => u.email === email && u.password === password, + ); + + if (!user) { + return res + .status(StatusCodes.UNAUTHORIZED) + .json({ message: "Invalid credentials" }); + } + + global.user_id = user; + + res.status(StatusCodes.OK).json({ + name: user.name, + email: user.email, + }); +}; + +export const logoff = async (req, res) => { + global.user_id = null; + res.sendStatus(StatusCodes.OK); +}; diff --git a/mentor-guidebook/sample-answers/assignment3/middleware/error-handler.js b/mentor-guidebook/sample-answers/assignment3/middleware/error-handler.js new file mode 100644 index 0000000..0026a56 --- /dev/null +++ b/mentor-guidebook/sample-answers/assignment3/middleware/error-handler.js @@ -0,0 +1,17 @@ +import { StatusCodes } from "http-status-codes"; + +const errorHandlerMiddleware = (err, req, res, next) => { + console.error( + "Internal server error: ", + err.constructor.name, + JSON.stringify(err, ["name", "message", "stack"]), + ); + + if (!res.headersSent) { + return res + .status(StatusCodes.INTERNAL_SERVER_ERROR) + .json({ message: "An internal server error occurred." }); + } +}; + +export default errorHandlerMiddleware; diff --git a/mentor-guidebook/sample-answers/assignment3/middleware/not-found.js b/mentor-guidebook/sample-answers/assignment3/middleware/not-found.js new file mode 100644 index 0000000..f02173d --- /dev/null +++ b/mentor-guidebook/sample-answers/assignment3/middleware/not-found.js @@ -0,0 +1,9 @@ +import { StatusCodes } from "http-status-codes"; + +const notFoundMiddleware = (req, res) => { + return res + .status(StatusCodes.NOT_FOUND) + .json({ message: `You can't do a ${req.method} for ${req.url}.` }); +}; + +export default notFoundMiddleware; diff --git a/mentor-guidebook/sample-answers/assignment3/routes/userRoutes.js b/mentor-guidebook/sample-answers/assignment3/routes/userRoutes.js new file mode 100644 index 0000000..e24619f --- /dev/null +++ b/mentor-guidebook/sample-answers/assignment3/routes/userRoutes.js @@ -0,0 +1,10 @@ +import express from "express"; +import { logon, register, logoff } from "../controllers/userController.js"; + +const router = express.Router(); + +router.post("/register", register); +router.post("/logon", logon); +router.post("/logoff", logoff); + +export default router; diff --git a/mentor-guidebook/sample-answers/assignment3/week-3-middleware/app.js b/mentor-guidebook/sample-answers/assignment3/week-3-middleware/app.js new file mode 100644 index 0000000..5fd976d --- /dev/null +++ b/mentor-guidebook/sample-answers/assignment3/week-3-middleware/app.js @@ -0,0 +1,99 @@ +import express from "express"; +import path from "path"; +import { fileURLToPath } from "url"; +import { v4 as uuidv4 } from "uuid"; +import dogsRouter from "./routes/dogs.js"; +import { + ValidationError, + NotFoundError, + UnauthorizedError, +} from "./errors.js"; + +const __dirname = path.dirname(fileURLToPath(import.meta.url)); + +const app = express(); + +app.use((req, res, next) => { + req.requestId = uuidv4(); + res.setHeader("X-Request-Id", req.requestId); + next(); +}); + +app.use((req, res, next) => { + const timestamp = new Date().toISOString(); + console.log( + `[${timestamp}]: ${req.method} ${req.path} (${req.requestId})`, + ); + next(); +}); + +app.use((req, res, next) => { + res.setHeader("X-Content-Type-Options", "nosniff"); + res.setHeader("X-Frame-Options", "DENY"); + res.setHeader("X-XSS-Protection", "1; mode=block"); + next(); +}); + +app.use(express.json({ limit: "1mb" })); +app.use(express.urlencoded({ limit: "1mb", extended: true })); + +app.use( + "/images", + express.static(path.join(__dirname, "public/images")), +); + +app.use((req, res, next) => { + if (req.method !== "POST") { + return next(); + } + const contentType = req.get("Content-Type") || ""; + const isJson = contentType.toLowerCase().includes("application/json"); + if (!isJson) { + return res.status(400).json({ + error: "Content-Type must be application/json", + requestId: req.requestId, + }); + } + next(); +}); + +app.use("/", dogsRouter); + +app.use((err, req, res, next) => { + const statusCode = err.statusCode || 500; + + if (statusCode >= 400 && statusCode < 500) { + if (err instanceof ValidationError) { + console.warn("WARN: ValidationError", err.message); + } else if (err instanceof UnauthorizedError) { + console.warn("WARN: UnauthorizedError", err.message); + } else if (err instanceof NotFoundError) { + console.warn("WARN: NotFoundError", err.message); + } else { + console.warn(`WARN: ${err.name}`, err.message); + } + } else { + console.error("ERROR: Error", err.message); + } + + const bodyError = + statusCode === 500 ? "Internal Server Error" : err.message; + + res.status(statusCode).json({ + error: bodyError, + requestId: req.requestId, + }); +}); + +app.use((req, res) => { + res.status(404).json({ + error: "Route not found", + requestId: req.requestId, + }); +}); + +if (!process.env.VITEST) { + app.listen(3000, () => console.log("Server listening on port 3000")); +} + +export default app; diff --git a/mentor-guidebook/sample-answers/assignment3/week-3-middleware/dogData.js b/mentor-guidebook/sample-answers/assignment3/week-3-middleware/dogData.js new file mode 100644 index 0000000..c112d3e --- /dev/null +++ b/mentor-guidebook/sample-answers/assignment3/week-3-middleware/dogData.js @@ -0,0 +1,54 @@ +export default [ + { + name: "Sweet Pea", + sex: "Female", + color: "Brown", + weight: "55 lbs", + breed: "Labrador Retriever", + age: 3, + story: "Rescued from a local shelter", + goodWithKids: true, + goodWithDogs: true, + goodWithCats: false, + status: "available", + }, + { + name: "Luna", + sex: "Female", + weight: "65 lbs", + color: "Black", + breed: "German Shepherd", + age: 5, + story: "Owner could no longer care for her", + goodWithKids: true, + goodWithDogs: false, + goodWithCats: false, + status: "available", + }, + { + name: "Max", + weight: "60 lbs", + sex: "Male", + color: "Golden", + breed: "Golden Retriever", + age: 2, + story: "Found as a stray", + goodWithKids: true, + goodWithDogs: true, + goodWithCats: true, + status: "pending", + }, + { + name: "Poppy", + weight: "9 lbs", + sex: "Female", + color: "Tricolor", + breed: "Dachshund", + age: 1, + story: "Rescued from a puppy mill", + goodWithKids: false, + goodWithDogs: true, + goodWithCats: false, + status: "available", + }, +]; diff --git a/mentor-guidebook/sample-answers/assignment3/week-3-middleware/errors.js b/mentor-guidebook/sample-answers/assignment3/week-3-middleware/errors.js new file mode 100644 index 0000000..74612a1 --- /dev/null +++ b/mentor-guidebook/sample-answers/assignment3/week-3-middleware/errors.js @@ -0,0 +1,23 @@ +export class ValidationError extends Error { + constructor(message) { + super(message); + this.name = "ValidationError"; + this.statusCode = 400; + } +} + +export class NotFoundError extends Error { + constructor(message) { + super(message); + this.name = "NotFoundError"; + this.statusCode = 404; + } +} + +export class UnauthorizedError extends Error { + constructor(message) { + super(message); + this.name = "UnauthorizedError"; + this.statusCode = 401; + } +} diff --git a/mentor-guidebook/sample-answers/assignment3/week-3-middleware/public/images/dachshund.png b/mentor-guidebook/sample-answers/assignment3/week-3-middleware/public/images/dachshund.png new file mode 100644 index 0000000..a50c8ae Binary files /dev/null and b/mentor-guidebook/sample-answers/assignment3/week-3-middleware/public/images/dachshund.png differ diff --git a/mentor-guidebook/sample-answers/assignment3/week-3-middleware/routes/dogs.js b/mentor-guidebook/sample-answers/assignment3/week-3-middleware/routes/dogs.js new file mode 100644 index 0000000..5fcf225 --- /dev/null +++ b/mentor-guidebook/sample-answers/assignment3/week-3-middleware/routes/dogs.js @@ -0,0 +1,41 @@ +import express from "express"; +import dogData from "../dogData.js"; +import { ValidationError, NotFoundError } from "../errors.js"; + +const router = express.Router(); + +router.get("/dogs", (req, res) => { + res.json(dogData); +}); + +router.post("/adopt", (req, res, next) => { + const { name, address, email, dogName } = req.body; + + if (!name || !email || !dogName) { + return next(new ValidationError("Missing required fields")); + } + + const dog = dogData.find((d) => d.name === dogName && d.status === "available"); + if (!dog) { + return next( + new NotFoundError("Dog not found or not available for adoption"), + ); + } + + return res.status(201).json({ + message: `Adoption request received. We will contact you at ${email} for further details.`, + application: { + name, + address, + email, + dogName, + applicationId: Date.now(), + }, + }); +}); + +router.get("/error", (req, res, next) => { + next(new Error("Test error")); +}); + +export default router;