fix log formatting with salt logger  #622

Activity

1. Mryashbhardwaj

   added

   bugSomething isn't workingSomething isn't working

   good first issueGood for newcomersGood for newcomers

   on Sep 27, 2022
2. camclark commented on Sep 27, 2022

   

   camclark

   on Sep 27, 2022

   More actions

   Hey team, interesting issue.
   Just stumbled on your project; mind if I have this issue assigned to me?
3. camclark commented on Sep 27, 2022

   

   camclark

   on Sep 27, 2022

   More actions

   Been playing around with the repo and I think I'm missing something to get local working.

   tldr: Followed local guide, but unable to get ./optimus serve to work from default configs. Under the impression localhost should work near out of the box.

   Followed steps as per: https://github.com/odpf/optimus#running-locally

   $ ./optimus version
   Client: v0.4.0-rc.6-next-17053a1
   
   $ ./optimus serve  
   WARN[0000] Shutting down server                         
   INFO[0000] Server shutdown complete                     
   Error: unable to create server: serve.ingress_host: required config missing
   🔥 unable to complete request successfully
   

   Since server isn't starting follow next steps within: https://odpf.github.io/optimus/docs/getting-started/configuration/

   $ ./optimus init   
   

   From here generated the optimus.yaml using values from this configuration page

   version: 1
   log:
     level: INFO
     format: ""
   host: localhost:9100
   project:
     name: local-project
     config: {}
   namespaces:
   - name: namespace1
     config: {}
     job:
       path: namespace1/jobs
     datastore:
     - type: bigquery
       path: namespace1/resources
       backup: {}
   

   attempt deploy:

   $ ./optimus deploy
   Initializing client plugins
   initialization finished!
   
   Registering project [local-project] to [localhost:9100]
   Error: Unable to reach optimus server at localhost:9100, this can happen due to following reasons:
   1. Check if you are connected to internet
   2. Is the host correctly configured in optimus config
   3. Is Optimus server currently unreachable
   🔥 unable to complete request successfully
   

   See server issue and scroll down on that page to: server config

   copy config.sample.yaml and uncomment as config.yaml

   version: 1
   
   # logging configuration
   log:
     # debug, info, warning, error, fatal - default 'info'
     level: info
   
   ########################################
   # SERVER CONFIG
   ########################################
   
   # for configuring optimus service
   serve:
    # port to listen on
    port: 9100
   
    # host to listen on
    host: localhost
   
    # this gets injected in compiled dags to reach back out to optimus service
    # when they run
    ingress_host: optimus.example.io:80
   
    # 32 char hash used for encrypting secrets
    # note: use a different one in production please
    app_key: Yjo4a0jn1NvYdq79SADC/KaVv9Wu0Ffc
   
    # database configurations
    db:
      # database connection string
      dsn: postgres://user:password@localhost:5432/database?sslmode=disable
   
      max_idle_connection: 5
      max_open_connection: 10
   
   # optimus supports multiple scheduler types
   scheduler:
    # name of the registered scheduler, default: airflow2
    name: airflow2
   
   
   # application telemetry
   telemetry:
   
    # configure the host:port used for
    # - exposing profiling metrics via pprof over /debug
    # - prometheus stats over /metrics
    profile_addr: ":9110"
   
    # jaeger collector address to send application traces
    jaeger_addr: "http://localhost:14268/api/traces"
   
   # resource managers for job dependency enrichment
   resource_managers:
   # - name: other_optimus_server
    type: optimus # currently, only support optimus type
    description: neighbor optimus server
    config:
      host: # host of other optimus server
      headers: # might necessary for authorization
   
   plugin:
     artifacts:
   #     # refer : https://github.com/hashicorp/go-getter
       - ../transformers/dist/transformers_0.1.0_macos_arm64.tar.gz
       - https://github.com/odpf/optimus/releases/download/v0.2.5/optimus_0.2.5_linux_arm64.tar.gz
   
   

   try serve again

   $ ./optimus serve
   
   2022/09/27 23:47:18 /Users/{myuser}/Documents/repos/open-source/optimus/internal/store/postgres/migration.go:165
   [error] failed to initialize database, got error failed to connect to `host=localhost user=user database=database`: dial error (dial tcp [::1]:5432: connect: connection refused)
   WARN[0000] Shutting down server                         
   INFO[0000] Server shutdown complete                     
   Error: unable to create server: error executing migration up: error initializing db client: error initializing db client: failed to connect to `host=localhost user=user database=database`: dial error (dial tcp [::1]:5432: connect: connection refused)
   🔥 unable to complete request successfully
   

   Please assist @Mryashbhardwaj
4. irainia commented on Oct 31, 2022

   

   irainia

   on Oct 31, 2022

   Member

   More actions

   @Mryashbhardwaj, this issue will be addressed by this PR. to be more specific, under this commit.
5. irainia commented on Oct 31, 2022

   

   irainia

   on Oct 31, 2022

   Member

   More actions

   @camclark , let me try to help you. in Optimus, we have two configurations (at the moment). one for client, which is named optimus.yaml and another is for server, named config.yaml. to run the server, make sure the config.yaml file is configured. I suggest you put in the same directory, otherwise you need to use flag -c (see --help flag).

   in case of your error:

   2022/09/27 23:47:18 /Users/{myuser}/Documents/repos/open-source/optimus/internal/store/postgres/migration.go:165
   [error] failed to initialize database, got error failed to connect to `host=localhost user=user database=database`: dial error (dial tcp [::1]:5432: connect: connection refused)
   WARN[0000] Shutting down server                         
   INFO[0000] Server shutdown complete                     
   Error: unable to create server: error executing migration up: error initializing db client: error initializing db client: failed to connect to `host=localhost user=user database=database`: dial error (dial tcp [::1]:5432: connect: connection refused)
   🔥 unable to complete request successfully

   try to check your database connection. it can be because you misconfigured it (like typos). or when reaching the database itself there is some problem, maybe connection issue or connection refused. if you don't have password to your database (such as for local), try to remove that, such as:

   ...
   serve:
     db:
       dsn: postgres://user@localhost:5432/db_name?sslmode=disable
       max_idle_connection: 5
       max_open_connection: 10
   ...