How to Use the Python Pickle Module for Persistent Object Storage with Examples Windows Tricks

Python’s `pickle` module, which enables the persistent storage of Python objects. In this article, we’ll explore the ins and outs of the `pickle` module, demonstrating its functionality through practical examples.

Table of Contents

1. Understanding Pickle.

The `pickle` module in Python is a part of the standard library and provides a mechanism for serializing and deserializing Python objects.

Serialization is the process of converting a Python object into a byte stream, while deserialization is the reverse process of reconstructing the object from the byte stream.
The primary purpose of the `pickle` module is to enable the persistent storage of Python objects. It allows you to save the state of your objects to a file, which can be later loaded to recreate the original objects.
This is particularly useful when you need to store complex data structures, such as dictionaries, lists, or custom objects, in a way that can be easily retrieved and used later.

2. Key Functions Provided by The `pickle` Module.

2.1 `pickle.dump(obj, file, protocol=None, *, fix_imports=True)`.

The pickle module’s dump function serializes the object `obj` and writes the byte stream to the file-like object `file`.
The `protocol` parameter specifies the pickle protocol to use (0, 1, 2, 3, 4, or 5), with 4 and 5 available in Python 3.4 and later.
The `fix_imports` parameter determines whether to fix the names of Python modules during pickling.

2.2 `pickle.load(file, *, fix_imports=True, encoding=”ASCII”, errors=”strict”)`.

Reads a byte stream from the file-like object `file` and deserializes it to reconstruct the original object.
The `fix_imports` parameter is similar to the one in `pickle.dump()`.
The `encoding` and `errors` parameters control the encoding and error handling during unpickling.

2.3 `pickle.dumps(obj, protocol=None, *, fix_imports=True)`.

Similar to `pickle.dump()`, but returns a bytes object containing the serialized data instead of writing it to a file.

2.4 `pickle.loads(bytes_object, *, fix_imports=True, encoding=”ASCII”, errors=”strict”)`.

Similar to `pickle.load()`, but takes a bytes object containing the serialized data instead of reading from a file.

2.5 Caution.

It’s important to note that while the `pickle` module is powerful and convenient, there are security considerations.

Loading pickled data from untrusted or unauthenticated sources can lead to security vulnerabilities, as it may execute arbitrary code.
Therefore, caution should be exercised when using `pickle` in scenarios where data integrity and security are critical. In such cases, alternative serialization formats or additional security measures may be considered.

3. Python ‘pickle’ Module Use Cases.

Saving and restoring the state of a program: This is useful for games, simulations, and other programs that need to be able to resume from where they left off.

Caching data in memory: Pickle can be used to serialize data to disk and then load it back into memory when needed. This can be useful for speeding up programs that access the same data frequently.
Communicating between different processes or machines: Pickle can be used to serialize data to a byte stream and then send it over a network or write it to a file. This can be useful for distributed computing applications.
Storing data in a database: Pickle can be used to serialize data to a byte stream and then store it in a database. This can be useful for storing complex objects or objects that contain binary data.

Here are some specific examples of how the pickle module can be used:
A game might use pickle to save the player’s progress: This way, the player can resume the game from where they left off even if they quit and restart the game later.
A web application might use pickle to cache the results of expensive database queries: This way, the application can avoid having to run the queries every time a user requests the same data.

A distributed computing application might use pickle to send data between different processes: For example, a master process might use pickle to send tasks to worker processes.
A machine learning application might use pickle to store trained models: This way, the models can be loaded and used to make predictions on new data without having to retrain the models from scratch.

4. Basic Usage of Pickle.

To start using the `pickle` module, you need to import it:
```
import pickle
```

Serialization and deserialization are achieved through the `dump()` and `load()` functions, respectively:

import pickle

def pickle_unpickle_built_in_object():
    # Serialization
    data_to_store = {'name': 'John', 'age': 30, 'city': 'New York'}
    with open('data.pkl', 'wb') as file:
        pickle.dump(data_to_store, file)

    # Deserialization
    with open('data.pkl', 'rb') as file:
        loaded_data = pickle.load(file)

    print(loaded_data)

if __name__ == "__main__":
    pickle_unpickle_built_in_object()

When you run the above Python code, it will create a file data.pkl file and save the dictionary object data in it, then it will load the file data and display the data on the screen like below.
```
{'name': 'John', 'age': 30, 'city': 'New York'}
```

5. Pickle & Unpickle Custom Objects.

Pickling is the process of converting a Python object into a byte stream, and unpickling is the process of reconstructing the original object from the byte stream. This is useful for saving and loading complex data structures or objects.

In this example, we’ll create a class called `Person` with attributes like name, age, and address. We’ll then create an instance of this class, pickle it to a file, and then unpickle it to recreate the object.

import pickle

class Person:
    def __init__(self, name, age, address):
        self.name = name
        self.age = age
        self.address = address

    def __str__(self):
        return f"Person(name={self.name}, age={self.age}, address={self.address})"


def pickle_unpickle_custom_object():
    # Creating an instance of the Person class
    person_instance = Person(name="John Doe", age=30, address="123 Main St, Cityville")

    # Pickling the object to a file
    pickle_file_path = "person_data.pkl"
    with open(pickle_file_path, 'wb') as file:
        pickle.dump(person_instance, file)

    # Unpickling the object from the file
    with open(pickle_file_path, 'rb') as file:
        loaded_person = pickle.load(file)

    # Displaying the original and loaded objects
    print("Original Person Object:", person_instance)
    print("Loaded Person Object:", loaded_person)


if __name__ == "__main__":
    pickle_unpickle_custom_object()

In this example: We define a `Person` class with attributes `name`, `age`, and `address`.
We create an instance of the `Person` class called `person_instance`.

We use the `pickle.dump()` function to serialize the object and write it to a file (person_data.pkl) in binary mode (`’wb‘`).
We use the `pickle.load()` function to read the serialized data from the file and reconstruct the original object.

When you run this code, you should see that the original and loaded objects are the same with the below output text.

Original Person Object: Person(name=John Doe, age=30, address=123 Main St, Cityville)
Loaded Person Object: Person(name=John Doe, age=30, address=123 Main St, Cityville)

Pickling and unpickling allow you to save and load complex objects in a convenient way, making it easy to store and retrieve data structures in your Python programs.

6. Example of Python Pickle `dumps()` and `loads()` Functions.

Example source code.

import pickle

def pickle_dumps_loads_function_example():
    # Example data (a dictionary)
    data_to_pickle = {
        'name': 'Alice',
        'age': 25,
        'city': 'Wonderland'
    }

    # Serialize (pickle) the data to a byte stream
    pickled_data = pickle.dumps(data_to_pickle)

    # Display the pickled data
    print("Pickled Data:")
    print(pickled_data)

    # Deserialize (unpickle) the data from the byte stream
    unpickled_data = pickle.loads(pickled_data)

    # Display the original and unpickled data
    print("\nOriginal Data:")
    print(data_to_pickle)

    print("\nUnpickled Data:")
    print(unpickled_data)


if __name__ == "__main__":
    pickle_dumps_loads_function_example()

Output.

Pickled Data:
b'\x80\x04\x950\x00\x00\x00\x00\x00\x00\x00}\x94(\x8c\x04name\x94\x8c\x05Alice\x94\x8c\x03age\x94K\x19\x8c\x04city\x94\x8c\nWonderland\x94u.'

Original Data:
{'name': 'Alice', 'age': 25, 'city': 'Wonderland'}

Unpickled Data:
{'name': 'Alice', 'age': 25, 'city': 'Wonderland'}

This code illustrates how to use the `pickle.dumps()` and `pickle.loads()` functions in Python to serialize and deserialize data, respectively. The example involves a dictionary, showcasing the process of converting Python objects to byte streams and reconstructing the original data from those streams.

7. Pickle and File Compression.

To further optimize storage and reduce file sizes, you can combine `pickle` with compression modules such as `gzip`. This is particularly beneficial when dealing with large datasets:

import pickle, gzip

def pickle_gzip_combine():
    # Serialization with compression
    data_to_store = {'name': 'Jane', 'age': 28, 'city': 'Los Angeles'}
    print(data_to_store)

    with gzip.open('compressed_data.pkl.gz', 'wb') as file:
        pickle.dump(data_to_store, file)

    # Deserialization with compression
    with gzip.open('compressed_data.pkl.gz', 'rb') as file:
        loaded_data = pickle.load(file)

    print(loaded_data)

if __name__ == "__main__":
    pickle_gzip_combine()