Status Update:
Root cause identified. Grafana SSO login fails with "User sync failed" after Keycloak authentication succeeds.
Keycloak authenticates the user correctly. Grafana then fails when creating its own local copy of that user. Grafana logs confirm this:
logger=user.sync level=error msg="Failed to create user" error="user not found" auth_module=oauth_generic_oauth
logger=authn.service level=error msg="Failed to run post auth hook" error="[user.sync.internal] unable to create user: user not found"The cause is duplicate email addresses. All users sourced from the GTAC LDAP provider share a single email address. Keycloak permits duplicate emails; Grafana requires every user to have a unique one. The first account created claims that email, and every subsequent user collides with it and is rejected.
Two secondary config issues were also found in grafana.ini:
login_attribute_path = usernamedoes not match Keycloak's standard claim (preferred_username), so Grafana was falling back to email as the username as well.login_maximum_lifetime_duration = 1mforces re-authentication after 1 minute (lowercasem= minutes). Confirming whether this was intentional.
We do not have access to the GTAC LDAP provider, so the fix is scoped entirely to Keycloak and Grafana. No LDAP changes are required.
Suggested Fix (summary):
- Remove the
emailclaim from the Keycloakgrafanaclient only, by changing theemailclient scope from Default to Optional. No other application and no user record is affected. - Configure Grafana to derive a unique email per user from the username via
email_attribute_path = join('', [preferred_username, '@grafana.local']). This value exists only in Grafana's internal user table. The real LDAP/Keycloak email is unchanged. - Correct
login_attribute_path,name_attribute_path, androle_attribute_pathto match the actual Keycloak claims. - Update existing Grafana accounts holding the shared email so they no longer block new SSO account creation.
- Once verified, disable the local login form and enable auto-login for SSO-only access.
Rejected alternatives: disabling Keycloak's Duplicate Emails realm setting (risks breaking the LDAP sync) and oauth_allow_insecure_email_lookup (could log different users into the same Grafana account).
Next Steps:
- Validate that
preferred_usernameis unique per user using the Keycloak client Evaluate tab. This is a gating check; the fix depends on it. — Owner: [name], ETA: [date] - Back up
grafana.iniandgrafana.db. - Apply the Keycloak client scope change and the
grafana.iniupdates in the test environment. - Clean up the existing Grafana accounts using the shared email.
- Restart Grafana and validate with two separate SSO users in isolated browser sessions.
- On successful validation, enable SSO-only login and confirm an SSO user holds the Grafana Admin role before disabling the local login form.
- Schedule the production change window once test results are confirmed.
Comments
Post a Comment