Extract photo spheres from E57

[loicroybon]

Hello, I'm trying to export photospheres as jpeg images from E57 file. I saw #4 but it doesn't work.
Do you have a working sample code that explains hox to proceed please ?

[loicroybon]

Hello
Up please :)

[giovanniborella]

I've had some luck doing it this way:

if __name__ == "__main__":

    e57 = pye57.E57(args.e57)
    imf = e57.image_file
    root = imf.root()

    print("File loaded successfully!")

    if not root['images2D']:
        print("File contains no 2D images. Exiting...")

    for image_idx, image2D in enumerate(root['images2D']):

        # Get intrinsic matrix
        pinhole = image2D['sphericalRepresentation']

        # Get picture from blob
        jpeg_image = pinhole['jpegImage']
        jpeg_image_data = np.zeros(shape=jpeg_image.byteCount(), dtype=np.uint8)
        jpeg_image.read(jpeg_image_data, 0, jpeg_image.byteCount())
        image = cv2.imdecode(jpeg_image_data, cv2.IMREAD_COLOR)
        
        cv2.imwrite(OUTFOLDER + '/' + str(image2D['name'].value()), image

[loicroybon]

Thanks I'd finally found this way too !

[SonaBIMDev]

Hi, Why I've got an error using sphericalRepresentation and not using pinholeRepresentation?

[spatialhast]

Hi,
how to extract photo spheres metadata (x,y,z coordinates, orientation)?

[anthonysbr]

Hi, how to extract photo spheres metadata (x,y,z coordinates, orientation)?

You can use the cartesianBounds and pose fields of the sphericalRepresentation element of each image.

x, y, z = pinhole['cartesianBounds']['xyz'] orientation = pinhole['pose']['rotation']

[Musaub7]

@spatialhast Did you have any luck solving it? I have the same problem currently and would appreciate some help.

[swell-d]

@spatialhast @Musaub7

    translation = image2D['pose']['translation']
    rotation = image2D['pose']['rotation']
    x = float(translation['x'].value())
    y = float(translation['y'].value())
    z = float(translation['z'].value())
    rx = float(rotation['x'].value())
    ry = float(rotation['y'].value())
    rz = float(rotation['z'].value())
    rw = float(rotation['w'].value())
    print(x, y, z, rx, ry, rz, rw)

[SonaBIMDev]

Have you already found a solution to extract panoramic image from e57 wich not contains embbeded image?

[swell-d]

Have you already found a solution to extract panoramic image from e57 wich not contains embbeded image?

I've got it. You could try to use rowIndex / columnIndex. I don't want to give a ready-made solution as I consider it my Know-How, but the direction of the search is roughly like this:

data = e57.read_scan(0, colors=True, row_column=True)
rgb = np.zeros((width + 1, height + 1, 3), np.uint8)
rgb[data["columnIndex"], data["rowIndex"], 0] = data["colorRed"]
rgb[data["columnIndex"], data["rowIndex"], 1] = data["colorGreen"]
rgb[data["columnIndex"], data["rowIndex"], 2] = data["colorBlue"]
image = Image.fromarray(rgb, mode='RGB')
image.show()

I use my solution hier: https://360-for-you.com/

[SonaBIMDev]

Hi @swell-d
Thx for your answer.
Is the image generated by your code the result of what you show us in your solution?
Because it seems to me that generating an image from point clouds requires the use of complex raster functions, like those shown in the link below.
https://iopscience.iop.org/article/10.1088/1755-1315/17/1/012160/pdf

[swell-d]

https://iopscience.iop.org/article/10.1088/1755-1315/17/1/012160/pdf

Applying rowColumn allows you to pull panoramas from e57 files generated, for example, in ReCap. It is not about synthesizing panoramas from points, it is only about extracting a panoramic photo.
Panoramas synthesized from points are generally of poor quality.

[SonaBIMDev]

Ok, I understand. I can imagine panorama extracted from points are poor. there must be a code extract somewhere...or I'm going to have to have a good understanding of the studies done to generate my own code..

[tajalagawani]

import numpy as np
import cv2
import pye57
import argparse
import os
import json
import logging

# Setup basic logging
logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(levelname)s - %(message)s')

def extract_and_save_images_and_metadata(e57_path, output_folder):
    logging.info("Loading E57 file...")
    e57 = pye57.E57(e57_path)
    imf = e57.image_file
    root = imf.root()

    logging.info("File loaded successfully.")

    if not root['images2D']:
        logging.warning("File contains no 2D images. Exiting...")
        return

    images_output_folder = os.path.join(output_folder, "images")
    metadata_output_folder = os.path.join(output_folder, "metadata")
    os.makedirs(images_output_folder, exist_ok=True)
    os.makedirs(metadata_output_folder, exist_ok=True)

    for image_idx, image2D in enumerate(root['images2D']):
        logging.info(f"Processing image {image_idx}...")

        pinhole = image2D['pinholeRepresentation'] if 'pinholeRepresentation' in image2D else image2D['sphericalRepresentation']
        jpeg_image = pinhole['jpegImage']
        jpeg_image_data = np.zeros(shape=jpeg_image.byteCount(), dtype=np.uint8)
        jpeg_image.read(jpeg_image_data, 0, jpeg_image.byteCount())
        image = cv2.imdecode(jpeg_image_data, cv2.IMREAD_COLOR)

        image_name = str(image2D['name'].value())
        image_path = os.path.join(images_output_folder, f"{image_name}.jpg")
        cv2.imwrite(image_path, image)
        logging.info(f"Saved image to {image_path}")

        if 'pose' in image2D:
            translation = image2D['pose']['translation']
            rotation = image2D['pose']['rotation']
            x = float(translation['x'].value())
            y = float(translation['y'].value())
            z = float(translation['z'].value())
            rx = float(rotation['x'].value())
            ry = float(rotation['y'].value())
            rz = float(rotation['z'].value())
            rw = float(rotation['w'].value())
            logging.info(f"Coords: x={x}, y={y}, z={z}, rx={rx}, ry={ry}, rz={rz}, rw={rw}")

            # Save pose information in metadata
            metadata = {
                'name': image_name,
                'translation': {'x': x, 'y': y, 'z': z},
                'rotation': {'x': rx, 'y': ry, 'z': rz, 'w': rw}
            }
        else:
            metadata = {'name': image_name}
            

        metadata_path = os.path.join(metadata_output_folder, f"{image_name}.json")
        with open(metadata_path, 'w') as f:
            json.dump(metadata, f, indent=4)
        logging.info(f"Saved metadata to {metadata_path}")

    logging.info("All images and metadata processed successfully.")


if __name__ == "__main__":
    parser = argparse.ArgumentParser(description='Extract and save 2D images and metadata from an E57 file.')
    parser.add_argument('e57', help='Path to the E57 file')
    parser.add_argument('--outfolder', default='extracted_data', help='Output folder for the extracted images and metadata')
    args = parser.parse_args()

    extract_and_save_images_and_metadata(args.e57, args.outfolder)

[dancergraham]

This seems a common issue / question people have - if someone could submit a code sample as a pull request that would be very helpful to other users of the library.