Decentralized File Storage with Python, IPFS, and Pinata: A Comprehensive Guide

Decentralized File Storage with Python, IPFS, and Pinata: A Comprehensive Guide

Harness the Power of Python and Pinata to Seamlessly Integrate with IPFS for Decentralized File Storage

ยท

5 min read

IPFS is a peer-to-peer protocol that enables decentralized file storage, retrieval, and sharing. Unlike traditional file systems that rely on centralized servers, IPFS uses a distributed network of nodes to store and deliver files. In this article, we will explore the concepts of IPFS and Pinata in more detail. We will delve into the benefits of using IPFS for decentralized file storage and discuss how Pinata complements this ecosystem as a centralized IPFS service. This article focuses on using IPFS with Python, using Pinata-based API, to learn more about IPFS, check this out

Let's Get Going

Libraries

  • Configure venv using python -m venv venv

  • Activate venv using source <path>/venv/bin/activate

  • Install Necessary Libraries pip install pinatapy-vourhey

Setting up Pinata

  • Move over to https://app.pinata.cloud/ and sign-up

  • Move to API KEY section and create a New Key

    Tick all that apply, for this article, we are using all Accesses. Later in article we will talk about a few too.

    Copy the Secrets appeared at screen and keep them safe.

Code

Libraries & Client

import os
import requests
from pinatapy import PinataPy

try:
    # Connect to the IPFS cloud service
    pinata_api_key="<>"
    pinata_secret_api_key="<>"
    pinata = PinataPy(pinata_api_key,pinata_secret_api_key)
    print("Connected")

except Exception as e:
    print(e)
    print("Error connecting to IPFS")

Output:

Connected

Functions to Upload Files & Objects

# Upload the file
def upload_file_to_ipfs(file_path):
    try:
        if os.path.exists(file_path):
            response = pinata.pin_file_to_ipfs(file_path)
            return response
        else:
            print("File not found")
    except Exception as e:
        print(e)
        print("Error uploading file to IPFS")



# Upload Object to IPFS
def upload_object_to_ipfs(dict_data):
    try:
        response = pinata.pin_json_to_ipfs(dict_data)
        return response
    except Exception as e:
        print(e)
        print("Error uploading object to IPFS")
  1. upload_file_to_ipfs(file_path) This component takes a file_path parameter, representing the path to the file you want to upload to IPFS. Here's how it works:

    • It checks if the file exists at the specified file_path.

    • If the file exists, it calls pinata.pin_file_to_ipfs(file_path) to upload the file to IPFS using Pinata.

    • The function returns the response received from Pinata, which typically includes information about the uploaded file, such as the IPFS hash.

  2. upload_object_to_ipfs(dict_data) This component takes a dict_data parameter, representing a Python dictionary object that you want to upload as a JSON object to IPFS. Here's how it works:

    • It calls pinata.pin_json_to_ipfs(dict_data) to upload the dictionary object as a JSON file to IPFS using Pinata.

    • The function returns the response received from Pinata, which typically includes information about the uploaded object, such as the IPFS hash.

The response received in both cases is called CID

A content identifier, or CID, is a label used to point to material in IPFS. It doesn't indicate where the content is stored, but it forms a kind of address based on the content itself. CIDs are short, regardless of the size of their underlying content.

CIDs are based on the contentโ€™s cryptographic hash. That means:

  • Any difference in the content will produce a different CID.

  • The same content added to two different IPFS nodes using the same settings will produce the same CID.

IPFS uses the sha-256 hashing algorithm by default, but there is support for many other algorithms.

Read more here

This response contains IPFS HASH, that can be used to retrieve objects

Get Pinned Objects from IPFS

# Get object from IPFS
def get_object_from_ipfs(ipfs_hash):
    try:
        response = requests.get("https://ipfs.io/ipfs/"+ipfs_hash)
        return response
    except Exception as e:
        print(e)
        print("Error getting object from IPFS")

get_object_from_ipfs(ipfs_hash) This component takes an ipfs_hash parameter, representing the IPFS hash of the object you want to retrieve. Here's a breakdown of its functionality:

  • It constructs a URL by appending the IPFS hash to the base URL "ipfs.io/ipfs". This URL represents the location where the object is stored in the IPFS network.

  • It uses the requests.get() method to send an HTTP GET request to the constructed URL.

  • The function returns the response received from the IPFS gateway, which typically contains the retrieved object's data.

Testing Function

if __name__ == "__main__":
    file_path = 'random.txt'
    result = upload_file_to_ipfs(file_path)
    print(result)


    #Get file from IPFS
    ipfs_hash = result['IpfsHash']
    response = get_object_from_ipfs(ipfs_hash)
    print(response.text)


    sample_dict = {
        "name": "John",
        "age": 30,
        "city": "New York"
    }
    result = upload_object_to_ipfs(sample_dict)
    print(result)
    ipfs_hash = result['IpfsHash']
    response = get_object_from_ipfs(ipfs_hash)
    print(response.text)

Output:

--- Upload File & Get File Section---
{'IpfsHash': 'Qmew4L8Eq6rgNcvHy5o7Wo4UE4e7PLTz4pQrpZNVHBMTyn', 'PinSize': 22, 'Timestamp': '2023-06-10T01:07:54.638Z', 'isDuplicate': True}
Hi
Hello world

---Upload Object & Get Object Section---
{'IpfsHash': 'QmRfckH5Y3SL7JZiXVQVovfBjAkgrH1x2vZetNnbU3Scv1', 'PinSize': 50, 'Timestamp': '2023-06-10T01:10:08.857Z'}
{"name":"John","age":30,"city":"New York"}

Check Data in Use

print(pinata.user_pinned_data_total())

user_pinned_data_total() is a method provided by PinataPy that sends a request to the Pinata API to fetch the total amount of data the user has pinned in your Pinata account.

Output:

{'pin_count': 3, 'pin_size_total': 78, 'pin_size_with_replications_total': 78}

Conclusion

In summary, IPFS (InterPlanetary File System) and Pinata offer a powerful combination for decentralized file storage. IPFS revolutionizes file storage by providing a distributed and resilient network where files are identified and retrieved based on their content, ensuring data availability and censorship resistance. Pinata acts as a user-friendly gateway to IPFS, simplifying the process of interacting with the decentralized network. With Python and the PinataPy library, developers can easily upload files and objects to IPFS, retrieve data, and manage their pinned content. This combination opens up a world of possibilities for creating scalable, secure, and decentralized applications that leverage the benefits of IPFS and Pinata.

Did you find this article valuable?

Support Ankur by becoming a sponsor. Any amount is appreciated!

ย