Web APIs & Data formats

Client-Server Model

APIs

Pydantic Model

Data Mapping

Lab 4 - web service

Author

Ludovic Deneuville

🚧

https://www.insee.fr/fr/metadonnees/geographie/commune/35047-bruz

https://portail-api.insee.fr/catalog/api/5029cf12-e930-4d24-a9cf-12e9307d241d/doc?page=36a0f1a5-af70-4662-a0f1-a5af709662f4 curl -X ‘GET’
‘https://api.insee.fr/metadonnees/geo/commune/35047/ascendants’
-H ‘accept: application/json’

curl -X ‘GET’
‘https://api.insee.fr/metadonnees/geo/canton/3504/communes’
-H ‘accept: application/json’

Before you start

This tutorial combines explanations and code phases. The explanations should not take precedence over those of your teacher.

Concepts covered

Explore existing web services (Manual & Swagger)
Build a Data Mapper to consume an external API with a different schema
Create your own REST API using FastAPI
Implement security via Tokens and data validation via Pydantic

1 Client-Server principle

To understand how software systems communicate, let’s step out of the terminal and into a clothing store. Imagine you are a customer trying to interact with a shop assistant; this simple interaction is the perfect metaphor for the Client-Server relationship and the world of APIs.

sequenceDiagram
    participant Client as Customer (Client)
    participant API as Shop Assistant (API)
    participant Server as Warehouse (Server)

    Note over Client, Server: READ Operations (Filtering)
    
    Client->>API: "Show me all underwear" (GET /underwear)
    API->>Server: Query all items
    Server-->>API: [Boxer, Briefs, Panties]
    API-->>Client: 200 OK (List of items)

    Client->>API: "I want pink briefs" (GET /underwear?type=briefs&color=pink)
    API->>Server: Filter collection
    Server-->>API: Not Found
    API-->>Client: 404 Not Found: "We don't carry pink briefs!"

But wait! A shop is not just for looking. If you want to actually do something-like adding new stock or changing prices-you need more than just a “viewing” request. You need to perform Write Operations.

sequenceDiagram
    participant Client as Customer (Client)
    participant API as Shop Assistant (API)
    participant Server as Warehouse (Server)

    Note over Client, Server: WRITE Operations (Targeting specific IDs)

    Client->>API: "Add a Rainbow Panties" (POST /underwear)
    Note right of Client: Body: {"type": "panties", "color": "rainbow"}
    API->>Server: Insert new record
    Server-->>API: Created (ID: 101)
    API-->>Client: 201 Created: "Added to catalog!"

    Client->>API: "Update Gold Boxer price" (PUT /underwear/99)
    Note right of Client: Body: {"price": 999.99}
    API->>Server: Update item 101
    Server-->>API: Success
    API-->>Client: 200 OK: "Price updated. French quality!"

    Client->>API: "Delete green briefs" (DELETE /underwear/42)
    API->>Server: Remove item 42
    Server-->>API: Deleted
    API-->>Client: 204 No Content: "Bye bye green briefs."

Now that anyone can add or delete items, our shop is a mess! We can’t just let anyone walk in and change everything. We need to implement Security and Permissions to protect our inventory.

sequenceDiagram
    participant Client as Customer (Client)
    participant API as Shop Assistant (API)
    participant Server as Warehouse (Server)
    Note over Client, Server: Security & Permissions

    Client->>API: "Get VIP items" (GET /vip-items)
    Note right of Client: Missing Token
    API-->>Client: 401 Unauthorized: "Reserved for members!"

    Client->>API: "Delete an item" (DELETE /underwear/101)
    Note right of Client: x-auth-token: r2D2c3Po Valid, but wrong Role
    API-->>Client: 403 Forbidden: "You are a customer, not the manager!"

2 Exploring the Web (The Manual Way)

Before writing code, you must understand how to “talk” to a server. A web service is just an application waiting for HTTP requests.

2.1 Reminder

In the first lab, we explained the difference between an API and an interface.

The Front-end is the “face” running on the user’s device, while the Back-end is the “brain” running on a remote server.

The UI is what the user sees; user actions are sent to the back-end via the API.

Concept	Primary User	Synonyms	Main Role
API	Machines / Software	Webservice, Endpoint	Enables two software systems to communicate
Back-end	Server / Database	Server-side	The server-side logic, API implementation.
UI (User Interface)	Human	Graphical Interface	The visual elements (buttons, menus) that a user interacts with
Front-end	Human	Client	The client-side. Code running in the browser that renders and manages the UI

2.2 Quick Browser Check

The simplest way to interact with an API is via your browser (Firefox, Chrome, etc.). However, browsers are limited to GET requests (read data).

Lets explore https://swapi.info/api/

What URL displays all the characters?
- What is the data structure?
What URL displays only the character Maxe Windu?
- What is the data structure?
What is Mace Windu’s home planet?
Find all planets visited in “The Empire Strikes Back”

🚧 example with parameters ?limit…

2.3 Professional API Clients

To perform more complex actions (such as POST, PUT, or DELETE), a web browser is not enough. You will need a specialized tool to build your requests and inspect the responses.

Depending on your preference, you can use one of the following methods:

2.3.1 Desktop Applications

These are powerful tools that you download and install:

2.3.2 Online Tools

No installation required. Perfect for a quick test directly in your browser.

2.3.3 Command Line

If you prefer staying in the terminal, you can use cURL.

It is available on almost all operating systems.

2.3.4 Interactive Documentation (Swagger)

If the web service provides it, always check the Swagger/OpenAPI page (usually at the /docs endpoint).

It is the best way to see the available endpoints, the required parameters, and even test them directly through a web interface.

3 Exploring your project’s API

Modern APIs provide a “manual” that is interactive: Swagger (OpenAPI).

3.1 Using Swagger

Your current application already has a web service running.

Launch the Back-end
Find the url of the swagger

Tip

You’ve already done this in the first Lab.

List all players
- Select the endpoint ➡️ Try it out ➡️ Execute
Delete the player with the highest ID

Up until now, you have used GET requests to ask for information (the same goes for DELETE). But to create or modify data, you must provide information. This is called the Request Body.

When using POST or PUT, you don’t just send a URL; you send a JSON object containing the details of the resource.

{
  "username": "new_player",
  "email": "new@player.com",
  "elo": 1200
}

Create a new player
Update player junior
- add 100 points to its elo, change suffix mail from io to fr
Play a coin flip between batricia and gilbert

Note

With each request, you’ll notice the curl command if you want to run it from the service’s terminal.

It’s all well and good to marvel at all that JSON code, but it’s usually machines and programs that consume it.

4 The Client Challenge (Data Mapping)

Now, we will move to a more advanced task. Your Back-end are going to act as a Client. You need to fetch game data from an external source and integrate it into your engine.

The Schema Mismatch

The external API does not use your internal names. It is a common real-world problem. You must bridge the gap!

4.1 The Mission

This part is an example to retrieve peoples from star wars API. No questions.

We will use package requests.

➡️ Step1: Call the web service

Run the query as you would with another tool and store the response in a variable.

import requests

response = requests.get(url="https://swapi.info/api/people/")

➡️ Step2: Check response

Then, see if the answer is useful.

If everything goes well, you can retrieve the JSON content.

if response.status_code != 200:
    raise Exception(f"Cannot reach (HTTP {response.status_code}): {response.text}")
else:
    raw_json = response.json()
    print(json.dumps(raw_json, indent=2))  # Pretty print

➡️ Step3: Convert into business objects

Imagine you have a People (name, birth, hair, bmi) business object.

You will have to iterate among all elements to create People:

peoples = []

for elt in raw_json:
    # Create an object
    p = People(
        name=elt["name"],
        birth=elt["birth_year"],
        hair=elt["hair_color"],
        bmi=compute_bmi(elt.get("mass"), elt.get("height"))
    )

    # If it succeed, add to the list
    if p:
      peoples.append(p)

def compute_bmi(mass_raw, height_raw) -> float:
    """Calculates BMI based on weight and height in centimeters."""
    try:
        mass = float(mass_raw)
        height = float(height_raw) / 100
        return round(mass / (height**2), 2)
    except (ValueError, TypeError, ZeroDivisionError):
        return None

Note

You can use some fields directly as attributes, and others as BMI (Body Mass Index) you won’t find it directly, you have to compute it.

🎉 You have converted a dictionary list into a list of business objects that can be used directly by your methods.

4.2 Your turn

🚧 TODO: Give a link, and simple procedure

Create a client folder and create file game_client.py

This client must fetch data from an external API and transform it into your Game business objects.

Below is the data available on this external API:

/games

[
  {
    "id": "1",
    "players_list": ["Alice", "Bob"],
    "winner_name": "Alice",
    "location_name": "Arena 1",
    "duration_seconds": 120,
    "mode_type": "coinflip"
  },
  {
    "id": "2",
    "players_list": ["Eve", "Frank"],
    "winner_name": "Eve",
    "location_name": "Secret Cave",
    "duration_seconds": 300,
    "mode_type": "dice"
  }
]

This is a list [element1, element2] of dictionaries {key1: value1, key2: value2}.

Important

The goal is to convert this dictionary list into a Game list by:

iterating through each element of the list one by one
creating a Game object from the available data in the dictionary

⚠️ A dictionary is not an object, so attributes are not required for every element.

Create a function get_games() -> list[Game]
Call the endpoint
Check response
Convert JSON response into a list of Game

In this section, you were a client of an API; next you’ll become the provider by creating your own endpoints.

5 Building your own API

Finally, you will become the Server. You will expose your Game data via a professional REST API using FastAPI.

5.1 The most simple API

Example, no question.

Below is the code and command to create and launch the simplest possible API using FastAPI:

➡️ Step1: Create the API

main.py

from fastapi import FastAPI

app = FastAPI()                                 # 1. Create the application instance

@app.get("/")                                   # 2. Define a root endpoint
async def read_root():
    return {"message": "Hello World"}

➡️ Step2: Launch the Server

Launch it with command: uvicorn main:app --host 0.0.0.0 --port 5000 --reload

Note

uvicorn: The ASGI server that runs your application
main:app: Refers to your file main.py and the FastAPI instance inside it app
–host 0.0.0.0: Makes the server accessible on your local network
–port 5000: Sets the port the API will listen on
–reload: it automatically restarts the server whenever you save a file

➡️ Step3: Query the API

Using curl -X GET http://127.0.0.1:5000/ or another client.

Expected output:

{"message": "Hello World"}

A production-ready API does much more than just return a single message; it uses various HTTP methods, handles query parameters, and navigates structured URL paths to interact with complex data.

5.2 Your First Endpoint

Open file main.py

The Application Entry Point

What’s in this file? It is the Orchestrator of your application:

Routing Assembly: It connects modular Routers (Controllers) to the main FastAPI instance
Utility Management: It provides administrative endpoints like database resets or documentation redirects
Server Execution: It launches the ASGI server (Uvicorn) to handle incoming HTTP requests

You’ll notice that every endpoint prefixed with /game will be handled in the game_controller.py file.

Open file game_controller.py
Add this endpoint below and test it in the swagger:

@router.get("/", tags=["Games"])
async def get_mock_games():
    return [
        {"id_game": 1, "mode": "coinflip", "winner": "Miguel"},
        {"id_game": 2, "mode": "dice", "winner": "Batricia"}
    ]

OK, that’s a start. Now, let’s aim to display all Games played by a specific Player. We’ll proceed step by step modifying the previous method.

Add parameter id_player to the method
- to have this endpont: GET /game?id_player=?
- for your first try, just add this ID into the returned JSON (new key)

If you want this parameter to be optional, you must specify a default value for it in the method signature (e.g., None).

Tip

A better way is to use a hierarchical URI that semantically expresses that games are a sub-resource belonging to a specific player.

player_controller.py

# GET /player/{id_player}/games
@app.get("/players/{id_player}/games", response_model=list[GameModel])
def get_games(id_player, service: game_service=Depends(get_game_service)):
    if ... # Player don't exists
        raise ...
    return ... # Call a service

Finally, instead of returning raw JSON data, let’s now retrieve the actual list of games played by the player with that ID.

Do you have a DAO that provides a list of games by player?
- If not, implement one.
Do you have a service that provides a list of games by player?
- If not, implement one.
Since it’s no longer a mock, rename the controller method

Here is the route:

sequenceDiagram
    participant Client
    participant Controller as GameController
    participant Service as GameService
    participant DAO as GameDAO
    participant DB as Database

    Client->>Controller: GET /game?id_player=1
    Note over Controller: Validates request & params
    Controller->>Service: find_all_by_player(1)
    Service->>DAO: find_all_by_player(1)
    DAO->>DB: SELECT * <br> FROM game <br> WHERE id_player = 1
    DB-->>DAO: Returns rows
    DAO-->>Service: list[Game]
    Service-->>Controller: list[Game]
    Controller-->>Client: 200 OK (JSON list)

Great, it works.

But do you notice anything that seems wrong?

What if our Game table contained confidential information?

How can we ensure that it isn’t exposed?

Fortunately, there is a solution for managing data input and output!

5.3 Data Validation

When you expose an endpoint, you don’t have to reveal all the data you have. It may be a good idea to hide passwords, tokens, and so on…

Conversely, when your endpoint receives data (e.g., form), you cannot trust the client. A user might send a string where you expect an integer, or forget a mandatory field. To prevent this “garbage” from entering your system, we use Pydantic models.

A Pydantic model acts as a Data Contract: it defines exactly what the data should look like. If the incoming JSON doesn’t match the contract, FastAPI will automatically reject the request with a “422 Unprocessable Entity” error.

5.3.1 Define a Model

Example: in the player_model.py file, we define how a Player should look when communicating with the API:

from pydantic import BaseModel, EmailStr, Field

class PlayerModel(BaseModel):
    id_player: int | None = None
    username: str
    password: str = Field(..., min_length=35)
    elo: int
    email: EmailStr
    pokemon_fan: bool

5.3.2 Use it in the Controller

Let’s look at the create_player() method in player_controller.py.

@router.post("/", tags=["Players"])
async def create_player(p: PlayerModel, player_service=Depends(get_player_service)):
    # ...
    player = player_service.create(p.username, p.password, p.elo, p.email, p.pokemon_fan)

Here is the magic happening behind the scenes:

The Request: a client sends request to the endpoint with a JSON body: {"username": "bob", "email": "bob@email.com", ...}.
Validation: FastAPI looks at your PlayerModel. It checks:

Is username a string?
Is elo an integer?
Is email a valid email format?
Are all mandatory fields present?

Conversion: If everything is correct, FastAPI automatically transforms that JSON into a Python object named p.
Execution: You can then access the data easily using dot notation (p.username, p.email) to pass it to your PlayerService.

If the client sends an invalid email or forgets the elo, FastAPI will stop the request immediately and return a “422 Unprocessable Entity” error. Your service code is never even executed, protecting your business logic from bad data.

Open file game_model.py in folder schema

There’s already a model. Let’s see how it’s used.

Open file game_controller.py, and check parameters of method play_games()
- to find out who the current player is, it use the Token
- the opponent and the choice (heads or tails) are provided by the GameModel

OK, cool. We already have a model for Game, but let’s say we want to create an endpoint that returns all the games. If we use this model, the games returned will just be a list of opponents and options. Some information is missing.

5.4 Multiple Models

The more, the better.

A common mistake is to use the same model for every single operation. However, in a professional API, the data you receive is often different from the data you send or the data you use for specific actions.

Think about the Player logic. Using only one model creates several issues:

During Registration: You need the password, but you do not have an id_player yet (the database generates it).
During Login: You only need the username and password. Including the email or the elo is a waste of bandwidth and a security risk.

To solve this, we use different models for different contexts. This is called Separation of Concerns.

In player_model.py, we define two specialized contracts to match the specific needs of our API endpoints:

PlayerModel: The “Full” version. It contains all player attributes (email, elo, pokemon_fan, etc.).

Usage: It is used in player_controller.py within the create_player and update_player methods to ensure that all required profile information is validated.

PlayerLoginModel: The “Minimal” version. It is stripped down to only the essential fields required for authentication (username and password).

Usage: It is used in login_controller.py within the login method. This prevents the API from requiring or exposing unnecessary data like email or elo during a simple login attempt.

Let’s get back to the Games.

The goal will be to create an endpoint to display a player’s games.

Create a GameModel with these fields:
- id_game, game_mode, detail, timestamp
- for player1, player2 and winner: use another Model (player1: aPlayerModel)
Modify endpoint GET /game?id_player=? to use this model

Going Further

While using two models is a great start, professional APIs often use even more granular models to handle different stages of a resource’s lifecycle. You could further optimize this by implementing:

PlayerCreate vs PlayerUpdate: Currently, PlayerModel is used for both. However, when creating a player, you might want to require an email, but when updating a player, you might want to allow them to change only their elo without re-sending their username.
PlayerResponse: A model for sending data back to the client that explicitly excludes the password field, ensuring sensitive information never leaves the server.
PlayerSearch: A specialized model for filtering players (e.g., searching only by username or pokemon_fan status).

5.5 Security: Token-based Authentication

Bonus

A professional API is rarely wide open. To protect your data, you will implement Token-based authentication.

Instead of asking for a username and password on every single request, the client sends a “secret key” (the Token) in the HTTP Headers. If the token is missing or incorrect, the server rejects the request immediately.

How to implement it

You will implement a dependency that checks for a specific header. If the header is invalid, you must raise an HTTPException with a 401 Unauthorized status code.

The Security Contract: - Header Key: X-Auth-Token - Valid Value: secret-key-123

Example of a protected endpoint in your controller:

@router.post("/", tags=["Games"])
async def create_game(
    game: GameCreate, 
    service=Depends(get_game_service),
    token: str = Header(None)
):
    # Validate the token
    if token != "secret-key-123":
        raise HTTPException(status_code=401, detail="Invalid token")
    
    return service.create(game)

Add Security: Protect your POST and DELETE routes with the X-Auth-Token check.
Test with Swagger: Verify that a request without the token returns a 401.

Conclusion

This tutorial guided you through the complete lifecycle of building a professional web service. You progressed from understanding the Client-Server principle to implementing a robust REST API:

Being client of an external API
Creating your own API and own endpoints
Using Pydantic models to enforce strict data contracts and prevent “garbage” data
Implementing Token-based authentication to protect sensitive routes.

End of the Lab

Important

When you have finished coding, don’t forget to:

Push your code to GitHub
- If your service is terminated, all unpushed code is lost…
Pause or delete your datalab services
- to free up reserved resources