This is the CCF Data Reconciliation - sample in typescript.
The CCF network will be used to host a reconciliation service where different parties with membership (banks and data providers) will be able to submit their own data to be reconciled against "each other's data" in a confidential manner without exposing the data to other members in the network.
When a new record(s) is submitted through ingestion endpoints, the application will search the key-value store by the record's key; if this key does not exist, it will be added; otherwise, a vote is added to this record with a member ID and the submitted value.
The reconciliation application consists of three main parts: Data Ingestion, Data Reconciliation and Data Reporting.
- Data ingestion
- API Endpoint: allow members to submit their data to reconciled.
- Accept single or batch of records
- API Endpoint: allow members to submit their data to reconciled.
- Data reconciliation
- Data is compared across all members, all members' data carry equal weight to reach consensus.
- Reconciliation is on each record, not on the entire data set.
- Data reporting
- API Endpoint: Members will query for results
- Query by specific record by
a unique identifier
- Query all data
- Query by specific record by
- API Endpoint: Members will query for results
📂
├── docs Sample application documentation
│ └── adrs All Architecture decision records (ADR)
│
├── governance
│ └── constitution CCF network constitution files
│ └── nodes CCF network nodes configs
│ └── scripts All governance scripts
│ └── vote Contains proposal voting acceptance and rejection logic
│
└── src Application source code
│ └── endpoints Application endpoints
│ └── models Domain models
│ └── repositories Data repositories
│ └── services Domain services
│ └── utils utility classes
│
└── test
└── data-samples Data files for tests|demo
└── e2e-test Application end to end tests
└── unit-test Application unit tests
To get started and run the application locally, start with setting up the environment.
# setup the environment
git clone https://github.com/microsoft/ccf-app-samples # Clone the samples repository
code ccf-app-samples # open samples repository in Visual studio code
# In the VScode terminal window
cd data-reconciliation-app # Navigate to reconciliation sample folder
make build # Build and create the application deployment bundle
Now the environment is ready, and there are several scenarios that could be executed at this stage.
-
Run the application's e2e-tests in a sandbox environment in the interactive mode
make demo
-
Run the application's e2e-tests in a sandbox (simulated) environment
make test
-
Run the application's e2e-tests on a Docker Container running a virtual (simulated) environment
make test-docker-virtual
-
Run the application e2e-tests on a Managed CCF environment
-
Start a CCF network with 3 active members and 1 user using the sandbox and deploy the application to it, the application and network are ready to receive requests
make start-host
-
Run the application's unit tests
make unit-test
These are the main scenarios; more commands are defined in Makefile
and are described in the following section.
A Makefile provides a front-end to interact with the project. It is used both locally, during CI, and on GitHub Actions. This Makefile is self-documented, and has the following targets:
help 💬 This help message :)
build 🔨 Build the Application
build-virtual 📦 Build Virtual container image from Dockerfile
build-enclave 📦 Build Enclave container image from Dockerfile
start-host 🏃 Start the CCF network using Sandbox.sh
test 🧪 Test the Data Reconciliation Application in the sandbox
test-docker-virtual 🧪 Test the Data Reconciliation Application in a Docker sandbox
test-docker-enclave 🧪 Test the Data Reconciliation Application in a Docker enclave
test-mccf 🧪 Test the Data Reconciliation Application in a Managed CCF environment
e2e-test 🧪 Run end to end tests
unit-test 🧪 Run the Application unit-test
demo 🎬 Demo the Data Reconciliation Application
demo-code-change 🎬 Demo the Data Reconciliation Application Code Change
clean 🧹 Clean the working folders created during build/demo
The application testing strategy depends on two main types of testing to maintain code quality and coverage:
- Unit testing: to test the application’s business logic (domain models and services), please follow here
- End to end testing: to test application’s workflows from beginning to end, please follow here
cd data-reconciliation-app # Navigate to reconciliation sample folder
make unit-test # Run the unit tests
make test # Run the end-to-end(e2e) tests
It is also possible to run the tests in Demo mode. This can be achieved by running
cd data-reconciliation-app # Navigate to reconciliation sample folder
make demo # Run interactive end-to-end(e2e) tests
There is a guide here explaining what the demo shows. This will run the tests but require manual intervention so you can inspect the state of the network.