Events
API
Public API
The following API controllers are defined:
- AppController : publishes (store and forward) and retrieve app events
- EventsController : publishes (store and forward) and retrieve generic events
- SubscriptionController : creates, retrieves, validates and deletes event subscriptions
Private API
The API controllers listed below are exclusively for use within the Notification solution:
- StorageController : saves incoming events to persistent storage (database)
- InboundController : pushes events to events-inbound queue
- OutboundController : identify and authorize event subscribers and push event and subscriber details to events-outbound queue
- WebhookReceiverController : provides end point to support automated testing of subscriptions
Database
Events data is persisted in a PostgreSQL database.
Tables
Each table in the events schema is described in the table below, followed by a diagram showing the relation between the tables.
| Table | Description |
|---|---|
| events | Contains each cloud event and the registered time |
| subscription | Holds metadata for each event subscription |
Indexes
A number of indexes have been introduces to increased performance of the frequently used queries.
| index | table |
|---|---|
| events_pkey | events |
| idx_events_computed_time | events |
| idx_events_cloudevent_id | events |
| idx_events_cloudevent_subject | events |
| idx_events_cloudevent_alternativesubject | events |
| idx_events_cloudevent_source | events |
| idx_events_cloudevent_type | events |
| idx_events_cloudevent_time | events |
| idx_events_cloudevent_resource_sequenceno | events |
| idx_events_cloudevent_subject_time | events |
| eventssubscription_pkey | subscription |
| idx_gin_subscription_consumer | subscription |
| idx_subscription_subject_source_type | subscription |
| idx_btree_subscription_sourcefilterhash | subscription |
| idx_subscription_resourcefilter | subscription |
Azure Storage Queues
Azure Storage Queues have been set up to enable the .NET application and Azure Functions to send data for processing by a different service.
Logic for enqueueing elements is handled by EventsQueueClient and called from all relevant services.
The queues we operate with are
- events-registration
- events-inbound
- events-outbound
- subscription-validation
Azure Functions
Multiple Azure Functions have been set up to decouple the processing and pushing of events as well as subscription validation. All functions run based on Storage Queue triggers. An important principle for the functions is that each function should only have a single dependency to a logical action e.g. save to database, push to recipient etc. This way, if the action fails a retry can be done without any unexpected side effects.
- EventsRegistration : Pushes events to internal API endpoint that persists cloud event to database and on success pushes to internal API endpoint for inbound processing.
- EventsInbound : Pushes event to internal API endpoint that finds subscriptions with filters matching the cloud event.
- EventsOutbound POST cloud event to webhook endpoint associated with a subscription.
- SubscriptionValidation POST validation cloud event to user-defined webhook endpoint to check that it is ready to receive data. If so, a request is sent to internal API endpoint that updated the status in the database.
| Name | Trigger queue | Retry mechanism |
|---|---|---|
| EventRegistration | events-registration | default |
| EventsInbound | events-inbound | default |
| EventsOutbound | events-outbound | CustomQueueProcessorFactory |
| SubscriptionValidation | subscription-validation | CustomQueueProcessorFactory |
Dependencies
The microservice takes use of a range of external and Altinn services as well as .NET libraries to support the provided functionality. Find descriptions of key dependencies below.
External Services
| Service | Purpose | Resources |
|---|---|---|
| Azure Database for PostgreSQL | Hosts the database | Documentation |
| Azure API Management | Manages access to public API | Documentation |
| Azure Functions | Hosts the server less functions | Documentation |
| Azure Key Vault | Safeguards secrets used by the microservice | Documentation |
| Azure Kubernetes Services (AKS) | Hosts the microservice and cron jobs | Documentation |
| Azure Monitor | Telemetry from the application is sent to Application Insights | Documentation |
| Azure Storage | Hosts the queues and persists data in queues | Documentation |
Altinn Services
| Service | Purpose | Resources |
|---|---|---|
| Altinn Authorization | Authorizes access to the API | Repository |
| Altinn Register | Provides lookup of alternative subjects person and org | Repository |
.NET Libraries
The Events microservice and functions take use of a range of libraries to support the provided functionality.
| Library | Purpose | Resources |
|---|---|---|
| Azure.Storage.Queues | Used to send data to the storage queues | Repository, Documentation |
| CloudNative.CloudEvents.AspNetCore | Used for encoding and decoding cloud events | Repository, Documentation |
| AccessToken | Used to validate tokens in requests | Repository, Documentation |
| JWTCookieAuthentication | Used to validate Altinn token (JWT) | Repository, Documentation |
| Npgsql | Used to access the database server | Repository, Documentation |
| Yuniql | DB migration | Repository, Documentation |
A full list of NuGet dependencies is available on GitHub.
Testing
Quality gates implemented for a project require an 80 % code coverage for the unit and integration tests combined. xUnit is the framework used and the Moq library supports mocking parts of the solution.
Unit tests
The unit test projects are is available on GitHub.
Integration tests
No integration tests using external components haeve ben implemented for this solution. Look to unit test project for all tests related to Events.
Automated tests
The automated test project is available on GitHub
The automated tests for this micro service are implemented through Grafana’s k6. The tool is specialized for load tests, but we do use it for automated API tests as well. The test set is used for both use case and regression tests.
Use case tests
All use case workflows are available on GitHub
Use case tests are run every 15 minutes through GitHub Actions. The tests run during the use case tests are defined in the k6 test project. The aim of the tests is to run through central functionality of the solution to ensure that it is running and available to our end users.
Regression tests
Regression tests are hosted in Azure DevOps pipelines and triggered at the end of a release pipeline.
Hosting
Web API
The microservice runs in a Docker container hosted in AKS, and it is deployed as a Kubernetes deployment with autoscaling capabilities
The events application runs on port 5090.
See DockerFile for details.
Azure Functions
The Azure functions are hosted in a function app only containing event related functions.
Azure Storage
The Storage Queues are hosted in a geo-zone-redundant, general purpose v2 storage account.
Database
The database is hosted on a PostgreSQL flexible server in Azure.
Build & deploy
Web API
- Build and Code analysis runs in a Github workflow
- Build of the image is done in an Azure DevOps Pipeline
- Deploy of the image is enabled with Helm and implemented in an Azure Devops Release pipeline
Azure Functions
-Build of the Azure Function zip file is done in an Azure DevOps Pipeline
- Deploy of the Azure Functions is done in an Azure DevOps Release pipeline
Database
- Migration scripts are copied into the Docker image of the web API when this is build
- Execution of the scripts is on startup of the application and enabled by YUNIQL
Run on local machine
Instructions on how to set up the service on local machine for development or testing is covered by the README in the repository.