v1.1.2

Fixes

• fix(package.json): resolve an error with importing the default module
• fix(.ignore): add the package-lock.json file

Build

• build: replace build system babel to tsc

Full Changelog: v1.1.0...v1.1.2

v1.1.0

Fixes

• fix: rename S3TransportConfig.groupId to S3TransportConfig.group

Full Changelog: v1.0.4...v1.1.0

v1.0.4

Winston S3 Transport

Logs generated through Winston can be transferred to an S3 bucket using winston-s3-tranport.

Installation

The easiest way to install winston-s3-transport is with npm.

npm install winston-s3-transport

Alternately, download the source.

git clone https://github.com/stegano/winston-s3-transport.git

Example

[!] The bucket path is created when the log is first created.

// Example - `src/utils/logger.ts`
import winston from "winston";
import S3Transport from "winston-s3-transport";
import { v4 as uuidv4 } from "uuid";
import { format } from "date-fns";

const s3Transport = new S3Transport({
  s3ClientConfig: {
    region: "ap-northeast-2",
  },
  s3TransportConfig: {
    bucket: "my-bucket",
    groupId: (logInfo: any) => {
      // Group logs with user IDs and store them in memory. 
      // If the user ID does not exist, we will use the `anonymous` group.
      return logInfo?.message?.userId || "anonymous";
    },
    bucketPath: (groupId: string = "default") => {
      const date = new Date();
      const timestamp = format(date, "yyyyMMddhhmmss");
      const uuid = uuidv4();
      // The bucket path in which the log is uploaded. 
      // You can create a bucket path by combining `groupId`, `timestamp`, and `uuid` values.
      return `/logs/${groupId}/${timestamp}/${uuid}.log`;
    },
  },
});

export const logger = winston.createLogger({
  levels: winston.config.syslog.levels,
  format: winston.format.combine(winston.format.json()),
  transports: [s3Transport],
});

export default logger;

Create log using winston in another module

// Example - another module
import logger from "src/utils/logger";
...
// Create a log containing the field `userId`
logger.info({ userId: 'user001', ....logs });

Configuration

s3ClientConfig

This library is internally using @aws-sdk/client-s3 to upload files to AWS S3.

• Please see AWSJavaScriptSDK/s3clientconfig

s3TransportConfig

bucket*: string

• AWS S3 Bucket name

bucketPath*: ((groupId: string) => string) | string

• AWS S3 Bucket path to upload log files

groupId?: (<T = any>(logInfo: T) => string) | string (default: "default")

• Group ID to identify the log

dataUploadInterval?: number (default: 1000 * 20)

• Data upload interval(milliseconds)

fileRotationInterval?: number (default: 1000 * 60)

• File rotation interval(milliseconds)

maxDataSize?: number (default: 1000 * 1000 * 2)

• Max data size(byte)

Motivation

It was created to reduce unnecessary costs by sending logs to Winston and storing them in AWS S3 buckets, and by partitioning the log data into efficient structures when aggregating the vast amount of log data accumulated in AWS Athena.