Data Serialization
Serialization turns data held in memory (objects, lists, maps) into a sequence of bytes or text that can be saved or sent. Deserialization reads it back into memory.
Common formats
| Format | Kind | Typical use |
|---|---|---|
| JSON | Text | Web APIs, configuration, data exchange |
| XML | Text | Older enterprise APIs, document formats |
| YAML | Text | Human-edited configuration |
| CSV | Text | Tables, spreadsheet import and export |
| Protocol Buffers, Avro, MessagePack | Binary | Compact and fast service-to-service messages |
Text formats are easy to read and debug. Binary formats are smaller and faster but need tooling and, usually, a schema. See JSON Fundamentals for more on the most common one.
Round trip
# Python
import json
text = json.dumps({"id": 7, "name": "Ada", "tags": ["admin", "dev"]}) # object -> text
data = json.loads(text) # text -> object
// JavaScript
const text = JSON.stringify({ id: 7, name: "Ada", tags: ["admin", "dev"] });
const data = JSON.parse(text);
What does not survive
Formats support fewer types than programming languages do. Expect to handle:
- Dates and times: JSON has no date type, so send an ISO 8601 string in UTC and parse it explicitly. See Dates, Times, and Time Zones.
- Large and exact numbers: parsers may turn numbers into floating point, losing precision. Send ids and money as strings or integers of the smallest unit. See Number Representation.
- Missing, null and empty mean different things. Agree on which you use.
- Custom objects, functions and cyclic references need converting to plain data first.
- Use UTF-8 for text unless told otherwise.
Handling data safely
- Treat incoming data as untrusted. Validate its shape and values after parsing (Input Validation).
- Never deserialize untrusted data with formats that can create arbitrary objects, such as Python’s
pickleor native object serialization in several languages. This can lead to code execution. - Agree on a contract. Document field names and types, ideally with a schema (for example JSON Schema or OpenAPI).
- Plan for change. Adding optional fields is usually safe; renaming or removing one breaks existing readers. Version your format when you must break it.
- Expect unknown fields. Many JSON libraries ignore extra fields by default, while a strict mode can reject them. Readers should tolerate fields they do not know, so a newer writer does not break an older reader.
- Keep secrets out of serialized logs and files.
Try it
Write a small program that builds a structure containing a string, a number, a list and a date. Serialize it to JSON, save it to a file, read it back and compare the result with the original. Which parts came back as a different type?
Common mistakes
- Sending dates as local-time strings instead of ISO 8601 in UTC.
- Sending large ids or money as floating-point numbers.
- Deserializing untrusted data with formats that can create arbitrary objects, such as pickle.
- Renaming or removing a field without versioning.
Check yourself
- Why is deserializing untrusted pickle data dangerous?
- How should a date and a large id be sent in JSON?
- Why should readers tolerate fields they do not know?