reading a CSV from the object storage bucket

[JJDSNT]

Hi,
is there any simple example of reading/parsing a CSV file from the bucket using typescript sdk?

const response = await this.objectStorageClient.getObject(getObjectRequest);

i got back:

{"value":{},"opcRequestId":"gru-1:MQN9rKTo9H8Q_jG5m2_oW4CaxD6U9qC_kFoCdBKmoePd8WgtnNK5jkgKRC2L5HoZ","eTag":"91627f81-cb4c-432f-b034-6a3f9d7d29d3","contentLength":390119,"contentMd5":"PDC4JBZN3xgryS+BgDKlMw==","contentType":"text/csv","lastModified":"2023-08-05T14:53:39Z","storageTier":"Standard","versionId":"3d0b6685-c171-4b3a-892d-1cbc35ad25a8","opcMeta":{}}

an empty response.value

thanks in advance,

[AnuravModak]

So, for this particular query I didnt find any scope to do it in a simpler way if we go by the current codebase, now I may be wrong but I have gone through the code present in: https://github.com/oracle/oci-typescript-sdk/blob/d43f11e1e8fd720194a8dd312a62f4c6ba1d97a2/lib/objectstorage/lib/client.ts#L1875C3-L2052C4
So, u can find the implementation of objectStorageClient.getObject above.

So, my idea of doing it in a simpler way is proposing an ObjectStorageHelper class to encapsulate the Object Storage-related functionality:

import * as fs from 'fs';
import * as oci from 'oci-sdk';
import * as csv from 'fast-csv';


class ObjectStorageHelper {
  private objectStorageClient: oci.ObjectStorageClient;

  constructor() {
    // Initialize the Object Storage client (you might want to pass credentials, etc.)
    this.objectStorageClient = new oci.ObjectStorageClient({});
  }

  public async readAndParseCsvFile(
    namespaceName: string,
    bucketName: string,
    objectName: string
  ): Promise<void> {
    try {
      const getObjectRequest: oci.requests.GetObjectRequest = {
        namespaceName: namespaceName,
        bucketName: bucketName,
        objectName: objectName,
      };

      const response = await this.objectStorageClient.getObject(getObjectRequest);

      // Check if response contains a readable stream
      if (response.value) {
        // Pipe the readable stream from the OCI response to the CSV parser
        response.value.pipe(csv.parse({ headers: true }))
          .on('data', (row) => {
            // Process each row of the CSV file
            console.log(row);
          })
          .on('end', () => {
            console.log('CSV file parsing complete.');
          });
      } else {
        console.log('No content in the response.');
      }
    } catch (error) {
      console.error('Error reading or parsing CSV file:', error);
    }
  }
}

// Example usage
const objectStorageHelper = new ObjectStorageHelper();
objectStorageHelper.readAndParseCsvFile('your-namespace', 'your-bucket-name', 'your-csv-file.csv');

It handles the getObject request and pipes the readable stream to the CSV parser for processing.
The ObjectStorageHelper class provides a more focused and user-friendly interface for a specific use case, abstracting away complexity and handling errors in a way that suits the task of reading and parsing CSV files from OCI Object Storage. It promotes a cleaner and more maintainable code structure for this specific operation.

Now, every solution or approach need not be feasible all the times and the same is applicable to this case also, so the contributors are encouraged to provide clarification and approval, determining whether a wrapper class like this is necessary or if there exists a simpler way to accomplish the task. Please advise. Also if you think this is useful, let me know i will raise PR for this.

@jodoglevy @waruwaruwaru @y-chandra

[jyotisaini]

@JJDSNT While right now we do not have a way to read csv from the object storage, the Helper class above does seem like a feasible solution to me. @AnuravModak Can you create a PR for the same and our team can take a look and review it.