This document describes the services that are a part of the Infoworks application and their corresponding log location:
User Interface (UI):
This Component is responsible for all the operations related to the user interface. This service uses the Apricot package in the backend
Governor:
Governor manages the state of every Infoworks artifact, It takes care of resolving dependencies across Infoworks artifacts to ensure that no two conflicting jobs run at the same point in time
Hangman:
This service is responsible for job execution, It keeps polling the job queue and checks whether there are any jobs to be executed. Before submitting any job hangman will check whether there is any dependency in executing the job.
Rest API :
This is a Rest API component of Infoworks and all the Internal and external API requests are handled using this service. Rest API provides all the necessary endpoints for creating a complete data lake.
DT:
DT or Data Transformation service is responsible for all the Transformation use cases in Infoworks.
Monitoring Service:
Monitors the resource utilization of the system and application in the Infoworks edge node.
The service collects different metrics like CPU usage, memory usage, disk usage and application status and displays graphs on the dashboard
Provides both real-time and historic views of resource utilization of the system and application
Ingestion Service:
This service is responsible for all data ingestion-related operations like onboarding of tables, Metacrawl, and ingestion of tables.
Platform:
All Infoworks platform-related activities are handled by platform services like handling user authentication, running scheduled jobs, establishing the connectivity between different Infoworks components, and collecting metrics, etc.
Configuration Service :
This component reads the configuration placed in the conf files, User Interface, and passes it accordingly to the corresponding Infowork services.
Postgres:
It is the metadata component for Airlow/ Orchestrator.
RabbitMQ:
Infoworks uses RabbitMQ as a messaging queue to process tasks. This Works tightly coupled with Orchestrator components.
Orchestrator:
Orchestrator is used to provide a simple, easy to use, graphical interface to plan, schedule, execute, monitor, and report various steps within an entire business process
This will enable end-to-end automation of complex business processes with the ability to closely monitor and remediate issues whenever required.
It uses Airflow, Airflow internally depends on Postgres for maintaining the metadata for every workflow.
Nginx :
Infoworks uses Nginx as a reverse proxy service to route all the Internal / External requests to different Infoworks Services.
MongoDB:
All the Infoworks-related metadata is stored in Mongo DB, This comes in 2 different offerings:
Infoworks Managed: MongoDB will be installed locally and is managed by Infoworks
Atlas Mongo: All metadata will be stored and managed by the Atlas cluster.
Log Locations: