Explore

Billing Service

SYSTEM STORYLINE

We have a microservice called billing-service, built with Spring Boot. It’s part of a larger system:

api-gateway (external entry)

billing-service (Spring Boot, port 8090)

tax-service (another microservice)

Oracle DB for invoices

billing-service:

Uses Spring Boot auto-configuration with starter dependencies (web, data-jpa, actuator, logback).

Uses application.yaml (not application.properties).

Has Actuator enabled for /actuator/health, /actuator/metrics, /actuator/env, /actuator/threaddump.

Uses a custom Logback configuration (logback-spring.xml) to write logs to /var/log/billing-service.

Last night, a new release was deployed with:

New feature: TaxClient to call tax-service

Some refactoring of InvoiceService and BillingService

Changes in application.yaml (added some tax-related properties)

This morning, the support team finds:

billing-service is not healthy.

curl http://billing-host:8090/actuator/health returns 503 / connection refused.

Systemd / Docker logs show that the application exits during startup.

Your job as an L1/L2 support engineer is to read the logs, figure out what went wrong, and discuss where to look next.

FULL SPRING BOOT LOG

2025-11-18 09:02:11.204 INFO 14632 --- [ main] c.c.billing.BillingServiceApplication : Starting BillingServiceApplication v2.6.0 on app-node-01 with PID 14632 (/opt/billing-service/app.jar started by svcuser in /opt/billing-service)

2025-11-18 09:02:11.209 INFO 14632 --- [ main] c.c.billing.BillingServiceApplication : The following profiles are active: prod

2025-11-18 09:02:11.998 INFO 14632 --- [ main] o.s.boot.SpringApplication : No active profile set, falling back to 1 default profile: "default"

2025-11-18 09:02:12.014 INFO 14632 --- [ main] o.s.b.context.config.ConfigDataLoader : Loaded config file 'classpath:/application.yaml' (prod)

2025-11-18 09:02:12.015 INFO 14632 --- [ main] o.s.b.context.config.ConfigDataLoader : Loaded config file 'classpath:/application-default.yaml'

2025-11-18 09:02:12.423 INFO 14632 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat initialized with port(s): 8090 (http)

2025-11-18 09:02:12.439 INFO 14632 --- [ main] o.apache.catalina.core.StandardService : Starting service [Tomcat]

2025-11-18 09:02:12.440 INFO 14632 --- [ main] o.apache.catalina.core.StandardEngine : Starting Servlet engine: [Apache Tomcat/9.0.80]

2025-11-18 09:02:12.716 INFO 14632 --- [ main] o.s.b.w.servlet.ServletRegistrationBean : Servlet dispatcherServlet mapped to [/]

2025-11-18 09:02:13.087 INFO 14632 --- [ main] o.s.b.a.actuator.EndpointId : Exposing 12 endpoint(s) beneath base path '/actuator'

2025-11-18 09:02:13.088 INFO 14632 --- [ main] o.s.b.a.e.web.EndpointLinksResolver : Exposed actuator endpoints: [health, info, env, metrics, threaddump, loggers, httptrace, configprops, beans, mappings]

2025-11-18 09:02:13.542 INFO 14632 --- [ main] o.s.b.d.LoggingSystem : Initializing Spring Boot logging using Logback

2025-11-18 09:02:13.553 INFO 14632 --- [ main] c.q.l.core.rolling.RollingFileAppender : Log file: /var/log/billing-service/billing-service.log

2025-11-18 09:02:13.771 WARN 14632 --- [ main] ch.qos.logback.core.util.FileSize : Could not parse size value [10MB] for maxFileSize

2025-11-18 09:02:13.986 WARN 14632 --- [ main] c.c.b.config.LoggingConfiguration : Custom logback configuration loaded with warnings. Falling back to default values where necessary.

2025-11-18 09:02:14.352 INFO 14632 --- [ main] o.s.b.a.orm.jpa.DatabaseLookup : Unable to determine jdbc url from datasource

2025-11-18 09:02:14.361 INFO 14632 --- [ main] o.s.b.a.orm.jpa.JpaBaseConfiguration : Configuring JPA repository support for default data source

2025-11-18 09:02:14.894 INFO 14632 --- [ main] o.s.c.s.PostProcessorRegistrationDelegate$BeanPostProcessorChecker : Bean 'invoiceRepository' of type [com.company.billing.repository.InvoiceRepository] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)

2025-11-18 09:02:15.214 INFO 14632 --- [ main] o.s.b.a.AutoConfigurationPackages : Registering auto-configuration base packages: [com.company.billing]

2025-11-18 09:02:15.768 INFO 14632 --- [ main] o.s.s.concurrent.ThreadPoolTaskExecutor : Initializing ExecutorService 'applicationTaskExecutor'

2025-11-18 09:02:16.104 INFO 14632 --- [ main] o.s.b.a.e.web.ServletEndpointRegistrar : Registered '/actuator/health' to healthEndpoint

2025-11-18 09:02:16.104 INFO 14632 --- [ main] o.s.b.a.e.web.ServletEndpointRegistrar : Registered '/actuator/env' to environmentEndpoint

2025-11-18 09:02:16.104 INFO 14632 --- [ main] o.s.b.a.e.web.ServletEndpointRegistrar : Registered '/actuator/metrics' to metricsEndpoint

2025-11-18 09:02:16.104 INFO 14632 --- [ main] o.s.b.a.e.web.ServletEndpointRegistrar : Registered '/actuator/threaddump' to threadDumpEndpoint

2025-11-18 09:02:16.781 INFO 14632 --- [ main] o.s.b.f.s.DefaultListableBeanFactory : Overriding bean definition for bean 'billingService' with a different definition: replacing [Generic bean: class [com.company.billing.service.BillingService]; scope=singleton] with [Generic bean: class [com.company.billing.service.BillingService]; scope=singleton; defined in file [/opt/billing-service/classes/com/company/billing/service/BillingService.class]]

2025-11-18 09:02:17.092 WARN 14632 --- [ main] o.s.b.f.s.DefaultListableBeanFactory : Circular reference detected: com.company.billing.service.BillingService -> com.company.billing.service.InvoiceService -> com.company.billing.service.BillingService

2025-11-18 09:02:17.726 INFO 14632 --- [ main] o.s.jdbc.datasource.DriverManagerDataSource : Loaded JDBC driver: oracle.jdbc.OracleDriver

2025-11-18 09:02:18.015 INFO 14632 --- [ main] o.s.b.a.orm.jpa.DatabaseLookup : Unable to determine jdbc url from datasource

2025-11-18 09:02:18.017 INFO 14632 --- [ main] o.s.b.a.orm.jpa.JpaBaseConfiguration : Using 'default' datasource

2025-11-18 09:02:18.562 INFO 14632 --- [ main] o.h.jpa.internal.util.LogHelper : HHH000204: Processing PersistenceUnitInfo [name: default]

2025-11-18 09:02:19.118 INFO 14632 --- [ main] org.hibernate.dialect.Dialect : HHH000400: Using dialect: org.hibernate.dialect.Oracle12cDialect

2025-11-18 09:02:19.771 INFO 14632 --- [ main] o.h.e.t.j.p.i.JtaPlatformInitiator : HHH000490: Using JtaPlatform implementation: [org.hibernate.engine.transaction.jta.platform.internal.NoJtaPlatform]

2025-11-18 09:02:20.224 INFO 14632 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port(s): 8090 (http) with context path ''

2025-11-18 09:02:20.227 INFO 14632 --- [ main] c.c.billing.BillingServiceApplication : Started BillingServiceApplication in 9.412 seconds (JVM running for 10.328)

2025-11-18 09:02:20.229 INFO 14632 --- [ main] c.c.billing.BillingServiceApplication : Application startup complete, validating dependencies...

2025-11-18 09:02:20.697 ERROR 14632 --- [ main] o.s.b.d.LoggingFailureAnalysisReporter :

***************************

APPLICATION FAILED TO START

***************************

Description:

Parameter 0 of constructor in com.company.billing.service.InvoiceService required a bean of type 'com.company.billing.client.TaxClient' that could not be found.

The injection point has the following annotations:

- @org.springframework.beans.factory.annotation.Autowired(required=true)

Action:

Consider defining a bean of type 'com.company.billing.client.TaxClient' in your configuration.

2025-11-18 09:02:20.701 ERROR 14632 --- [ main] o.s.boot.SpringApplication : Application run failed

org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'invoiceService' defined in file [/opt/billing-service/classes/com/company/billing/service/InvoiceService.class]:

Unsatisfied dependency expressed through constructor parameter 0;

nested exception is org.springframework.beans.factory.NoSuchBeanDefinitionException:

No qualifying bean of type 'com.company.billing.client.TaxClient' available: expected at least 1 bean which qualifies as autowire candidate. Dependency annotations: {@org.springframework.beans.factory.annotation.Autowired(required=true)}

at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:799)

at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:228)

at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1350)

at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1196)

at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:556)

at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:516)

at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:324)

at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)

at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:322)

at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202)

at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:961)

at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:941)

at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:608)

at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:145)

at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:730)

at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:432)

at org.springframework.boot.SpringApplication.run(SpringApplication.java:308)

at org.springframework.boot.SpringApplication.run(SpringApplication.java:1306)

at org.springframework.boot.SpringApplication.run(SpringApplication.java:1295)

at com.company.billing.BillingServiceApplication.main(BillingServiceApplication.java:26)

Caused by: org.springframework.beans.factory.NoSuchBeanDefinitionException: No qualifying bean of type 'com.company.billing.client.TaxClient' available:

expected at least 1 bean which qualifies as autowire candidate. Dependency annotations: {@org.springframework.beans.factory.annotation.Autowired(required=true)}

at org.springframework.beans.factory.support.DefaultListableBeanFactory.raiseNoMatchingBeanFound(DefaultListableBeanFactory.java:1805)

at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1369)

at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1311)

at org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:887)

at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:791)

... 18 common frames omitted

2025-11-18 09:02:20.711 INFO 14632 --- [ main] o.s.s.concurrent.ThreadPoolTaskExecutor : Shutting down ExecutorService 'applicationTaskExecutor'

2025-11-18 09:02:20.712 INFO 14632 --- [ main] o.apache.catalina.core.StandardService : Stopping service [Tomcat]

2025-11-18 09:02:20.729 INFO 14632 --- [ main] o.s.b.a.actuator.logging.LogFileWebEndpoint : Stopped logging to file '/var/log/billing-service/billing-service.log' due to application shutdown

A. Touching all your topics

From this single log you can hit:

Spring Boot architecture & auto-config

Auto-configuration base packages, JPA, Tomcat, etc.

Config files: application.yaml

Lines where config is loaded.

Logging / Logback

Logback initialisation + size warning + custom config mention.

Actuator endpoints

Lines showing health/env/metrics/threaddump registration.

Common startup failures

Missing bean (TaxClient) and circular reference warning.

Practical debugging

Reading “APPLICATION FAILED TO START”, finding the real Caused by, classifying App vs DB vs Infra, deciding next steps.

B. Example discussion flow

You could walk them like this:

“What do you notice first?”

App name, version, port, profiles.

“Where do we see config files?”

Loaded config file 'classpath:/application.yaml' (prod)

Talk about YAML vs properties.

“Where do we see auto-configuration?”

JPA, datasource, dialect, auto-configuration packages.

“Where does logging come into picture?”

Logback init, rolling file appender, warning about maxFileSize (10MB).

“Where does Actuator show up?”

Exposing 12 endpoint(s), Registered '/actuator/health'...

“What’s this circular reference warning?”

Circular reference detected: BillingService -> InvoiceService -> BillingService

“Where does it actually die?”

APPLICATION FAILED TO START block about TaxClient missing.

Then the UnsatisfiedDependencyException / NoSuchBeanDefinitionException.

“App / DB / Infra?”

Missing bean + circular reference = App wiring / configuration.

“If you are L1, what 2–3 actions next?”

Check if TaxClient exists & is annotated (@Component/@Service).

Check if its package is under com.company.billing.

Check if any conditional properties (like tax.client.enabled) are missing in YAML.

Raise a clear ticket to dev team with the log snippet.

SOLUTION & EXPLANATION

We will go through the log in the same sequence as Spring Boot starts up, and identify every major issue, connecting each one to Spring Boot fundamentals: auto-config, YAML, logging, actuator, bean wiring, and common startup failures.

1. Spring Boot Application Starts — Normal Logs

Starting BillingServiceApplication …

Profiles active: prod

Loaded config file 'classpath:/application.yaml' (prod)

What this means

The application is running in prod profile.

Both application.yaml and application-default.yaml are loaded.

Nothing wrong here — normal.

Discussion

Profiles matter because beans and configs can change with profile.

YAML is being parsed correctly at this stage.

2. Tomcat Starts — Normal Web Boot

Tomcat initialized with port(s): 8090

dispatcherServlet mapped to [/]

Interpretation

Web layer is loading fine.

No port conflict (otherwise: BindException: Address already in use).

Nothing wrong yet.

3. Actuator Endpoints Load

Exposing 12 endpoint(s)…

Registered '/actuator/health'

Registered '/actuator/env'

Registered '/actuator/threaddump'

Interpretation

Actuator is enabled and exposing endpoints normally.

What this tells us

No actuator security misconfiguration at startup.

Spring Boot successfully mapped the endpoints.

4. Logging System Loads with a Warning

Initializing Spring Boot logging using Logback

Could not parse size value [10MB] for maxFileSize

Custom logback configuration loaded with warnings

❗ Problem #1 — Logback Misconfiguration

The Logback configuration contains:

But correct format is:

10MB ← (logback expects lowercase 'mb'? sometimes depends on version)

10MB

Logback throws this when the value is not parseable.

Does this break the application?

No (in this case). It only issues a warning and falls back to defaults.

👉 Discussion

Logging misconfiguration can cause apps not to start (e.g., missing custom appender classes).

In this case, the app continues anyway.

5. JPA + Datasource Initialization Issues

Unable to determine jdbc url from datasource

Using 'default' datasource

Problem #2 — Datasource Not Fully Configured

This line indicates Spring Boot does not detect the JDBC URL properly.

Possibilities:

YAML indentation issue

Wrong key names

Misplaced datasource block

Missing profile-specific override

Does this crash the app?

No — but it indicates configuration problems.

Discussion

If datasource is not set, JPA may still boot, but your app will fail later during a DB operation.

6. Auto-configuration Packages Registered

Registering auto-configuration base packages: [com.company.billing]

Want to print your doc?
This is not the way.

Try clicking the ··· in the right corner or using a keyboard shortcut (

CtrlP

) instead.