Trying to troubleshoot Saml2 SSO between Zulip and Plone

Changing it to "PasswordProtectedTransport" worked... almost. I now get a login prompt from the Zope server. But then I get an error when I try to move on.

It's a KeyError related to my SP
KeyError: u'https://zulipsp.mysite.com'

  Module ZPublisher.mapply, line 77, in mapply
  Module ZPublisher.Publish, line 48, in call_object
  Module dm.zope.saml2.idpsso.idpsso, line 188, in idpsso_logged_in
  Module dm.zope.saml2.idpsso.idpsso, line 118, in _okAuthnRequest
  Module dm.zope.saml2.idpsso.idpsso, line 144, in _make_authn_assertion
  Module dm.zope.saml2.role, line 204, in subject_from_member
  Module dm.zope.saml2.role, line 348, in get_role_descriptor
  Module dm.saml2.metadata, line 362, in metadata_by_id
  Module UserDict, line 40, in __getitem__
KeyError: u'https://zulipsp.mysite.com'

I'm off to look for what needs to be adjusted in my Plone site.

SAML2 requires that all participating (SAML) entities know one another. Corresponding entity descriptions are managed as content objects below the (SAML) authority. As I wrote earlier, dm.zope.saml2 is metadata based: the usual way to make a partner entity known is via an entity by url object: it takes the url for the entities metadata as parameter and does everything else automatically (based on the metadata description). Should your SP not provide describing metadata, the easiest thing is to describe it in a manually created metadata file and use a file url with entity by url.

yup... more progress.
I had to create an entry under my saml2auth with
id - https://zulipsp.mysite.com
url - https://zulipsp.mysite.com/saml/metadata.xml

The error is gone and now I have a failed to load key file error:

2021-05-29T15:17:10 ERROR Zope.SiteErrorLog 1622301430.80.20729305271 https://samldemo.alteroo.com/saml2idp/redirect
Traceback (innermost last):
  Module ZPublisher.Publish, line 138, in publish
  Module ZPublisher.mapply, line 77, in mapply
  Module ZPublisher.Publish, line 48, in call_object
  Module dm.zope.saml2.browser.role, line 43, in redirect
  Module dm.zope.saml2.browser.role, line 78, in _process
   - __traceback_info__: <samlp:AuthnRequest
  xmlns:samlp="urn:oasis:names:tc:SAML:2.0:protocol"
  xmlns:saml="urn:oasis:names:tc:SAML:2.0:assertion"
  ID="ONELOGIN_b2d86ca659e7ffecc007c0fcfa370b72af12d6a3"
  Version="2.0"
    ProviderName="SAML Zulip"
  IssueInstant="2021-05-29T15:14:26Z"
  Destination="https://mysite.com.com/saml2idp/redirect"
  ProtocolBinding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST"
  AssertionConsumerServiceURL="https://zulipsp.mysite.com/complete/saml/">
    <saml:Issuer>https://samlsp.mysite.com</saml:Issuer>
    <samlp:NameIDPolicy
        Format="urn:oasis:names:tc:SAML:1.1:nameid-format:unspecified"
        AllowCreate="true" />
    <samlp:RequestedAuthnContext Comparison="exact">
        <saml:AuthnContextClassRef>urn:oasis:names:tc:SAML:2.0:ac:classes:PasswordProtectedTransport</saml:AuthnContextClassRef>
    </samlp:RequestedAuthnContext>
</samlp:AuthnRequest>
  Module dm.zope.saml2.idpsso.idpsso, line 99, in handle_AuthnRequest
  Module dm.zope.saml2.idpsso.idpsso, line 138, in _okAuthnRequest
  Module dm.zope.saml2.role, line 116, in deliver_success
  Module dm.zope.saml2.role, line 154, in deliver
  Module dm.zope.saml2.role, line 259, in http_post
  Module dm.saml2.binding.httppost, line 28, in encode
  Module pyxb.binding.basis, line 539, in toxml
  Module pyxb.binding.basis, line 520, in toDOM
  Module dm.saml2.signature, line 311, in finalize
  Module dm.saml2.signature, line 154, in sign
  Module dm.zope.saml2.authority, line 406, in sign
  Module dm.zope.saml2.authority, line 312, in _get_signature_context
  Module dm.zope.saml2.authority, line 324, in _add_sign_keys
  Module dm.xmlsec.binding._xmlsec, line 181, in dm.xmlsec.binding._xmlsec.Key.load
ValueError: ('failed to load key from file', '/home/david/demo.saml2/var/instance/../../saml.key')

I checked and the key file is located at that location. My working guess right now is that the file is incorrectly formatted.

This is how I created the key file and certificates etc..

In the root of my buildout, I generated the key and certificate:

    openssl genrsa -des3 -out saml.key 2048
    openssl req -new -key saml.key -out saml.csr
    openssl x509 -req -days 365 -in saml.csr -signkey saml.key -out saml.crt

Then I created a der file:

openssl x509 -outform der -in saml.crt -out saml.der

I also created a p12 file, which I don't use, but it's there:

    openssl pkcs12 -export -out saml.p12 -inkey saml.key -in saml.crt -certfile saml.crt

After going back to the saml2auth settings for my Plone site, I traced it to the lack of a private key password. It is now added and the sign in flow proceeds without error.
image

But it simply takes me back to my Zulip login page :man_shrugging:
Everything indicates that the Plone site is working as an identify provider now. Now I'm figuring out the "last leg" on the Zulip/SP side of things..

The SP may require more information (than the user id) in attributes. An dm.zope.saml2 IDP will deliver attributes only if they are requested via the SP's metadata description. Consult the SP documentation for integration requirements.

I commented out the attributes:

SOCIAL_AUTH_SAML_ENABLED_IDPS = {
    ## The fields are explained in detail here:
    ##     https://python-social-auth.readthedocs.io/en/latest/backends/saml.html
    "mysite": {
        ## Configure entity_id and url according to information provided to you by your IdP:
        "entity_id": "saml2idp",
        "url": "https://mysite.com/saml2idp/redirect",
      ##  "attr_user_permanent_id": "login_name",
      ##      "attr_first_name": "first_name",
     ##      "attr_last_name": "last_name",
     ##       "attr_username": "email",
     ##       "attr_email": "email",
   
        "display_name": "Mysite",
   
    },
}

I'll start by trying to see what attributes are returned by Plone/collective.saml2 and then update it in the map.

Some additional info:
The logs of the Zulip server say:

Authentication failed: SAML login failed: ['invalid_response'] (There is no AttributeStatement on the Response)

A dm.zope.saml2 IDP will include an attribute statement only if known attributes are requested via the SP's metadata. The attribute statement then provides information for those attributes.

On the SP side I've updated the information for the request to request attributes:

SOCIAL_AUTH_SAML_ENABLED_IDPS = {
    ## The fields are explained in detail here:
    ##     https://python-social-auth.readthedocs.io/en/latest/backends/saml.html
    "saml2auth": {
        ## Configure entity_id and url according to information provided to you by your IdP:
        "entity_id": "saml2auth",
        "url": "https://mysite.com/saml2idp/redirect",
      "attr_user_permanent_id": "email",
      ##      "attr_first_name": "first_name",
     ##      "attr_last_name": "last_name",
          "attr_username": "email",
          "attr_email": "email",
   
        "display_name": "Mysite",
   
    },
}

My intention is to use the email address for the id and username. I expected this to trigger a request for attributes from the Plone collective.saml2 based idP.

But when I inspected the response from the IdP it definitely did not include the email attribute.
Here's the decoded and deflated response from my Plone site:

<?xml version="1.0" encoding="UTF-8"?>
<ns1:Response xmlns:ns1="urn:oasis:names:tc:SAML:2.0:protocol" xmlns:ns2="urn:oasis:names:tc:SAML:2.0:assertion" xmlns:ns3="http://www.w3.org/2000/09/xmldsig#" Destination="https://zulipsp.mysite.com/complete/saml/" ID="_93593693-ca36-48d0-936b-cb94864566f5" InResponseTo="ONELOGIN_692e274b158cc7b577ddd4abbc18669d6bc44763" IssueInstant="2021-05-31T13:29:45.216257Z" Version="2.0">
   <ns2:Issuer>saml2auth</ns2:Issuer>
   <ns1:Status>
      <ns1:StatusCode Value="urn:oasis:names:tc:SAML:2.0:status:Success" />
   </ns1:Status>
   <ns2:Assertion ID="_84886852-e970-4f60-81f3-924296ee71b1" IssueInstant="2021-05-31T13:29:45.211518Z" Version="2.0">
      <ns2:Issuer>saml2auth</ns2:Issuer>
      <ns3:Signature>
         <ns3:SignedInfo>
            <ns3:CanonicalizationMethod Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#" />
            <ns3:SignatureMethod Algorithm="http://www.w3.org/2000/09/xmldsig#rsa-sha1" />
            <ns3:Reference URI="#_84886852-e970-4f60-81f3-924296ee71b1">
               <ns3:Transforms>
                  <ns3:Transform Algorithm="http://www.w3.org/2000/09/xmldsig#enveloped-signature" />
                  <ns3:Transform Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#" />
               </ns3:Transforms>
               <ns3:DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1" />
               <ns3:DigestValue>EEV/F2pK6BrIZYYiaBFiGUJEvxI=</ns3:DigestValue>
            </ns3:Reference>
         </ns3:SignedInfo>
         <ns3:SignatureValue>HAvP0...JTAsxQ==</ns3:SignatureValue>
      </ns3:Signature>
      <ns2:Subject>
         <ns2:NameID Format="urn:oasis:names:tc:SAML:1.1:nameid-format:unspecified" NameQualifier="saml2auth">samlman</ns2:NameID>
         <ns2:SubjectConfirmation Method="urn:oasis:names:tc:SAML:2.0:cm:bearer">
            <ns2:SubjectConfirmationData InResponseTo="ONELOGIN_692e274b158cc7b577ddd4abbc18669d6bc44763" NotOnOrAfter="2021-05-31T13:34:45.213279Z" Recipient="https://zulipsp.mysite.com/complete/saml/" />
         </ns2:SubjectConfirmation>
      </ns2:Subject>
      <ns2:Conditions>
         <ns2:AudienceRestriction>
            <ns2:Audience>https://zulipsp.mysite.com</ns2:Audience>
         </ns2:AudienceRestriction>
      </ns2:Conditions>
      <ns2:AuthnStatement AuthnInstant="2021-05-31T13:29:45.211518Z">
         <ns2:AuthnContext>
            <ns2:AuthnContextClassRef>urn:oasis:names:tc:SAML:2.0:ac:classes:PasswordProtectedTransport</ns2:AuthnContextClassRef>
         </ns2:AuthnContext>
      </ns2:AuthnStatement>
   </ns2:Assertion>
</ns1:Response>

Are you suggesting that I don't need to do this for my attributes to work :point_down:?

To be perfectly honest... using 'email' and 'user_id' are educated guesses from inspecting some frontend code (not the best way to go about this).
If I do need to retrieve the information, it could be that I'm simply retrieving the wrong attributes.

For efficiency reasons, SAML metadata is cached. When you want to get metadata changes effective immediately, you must manually update it.

If a manual update does not work, check the SP's metadata. Note that attributes are identified by type and name, not by "friendly name". If the SP requests an attribute unsupported by the IDP, you will get a corresponding log message.

This is what I see in the logs on the Plone (IdP) side:

2021-05-31T15:21:11 WARNING pyxb.binding.content Multiple accepting paths for {urn:oasis:names:tc:SAML:2.0:assertion}AuthnContextType

No, I do not. Regarding attributes, there are two required things: the SP must specify which attributes it requests and the IDP must declare which attributes it supports (and how their value is computed). The first thing is done via the SP''s metadata description; the second via IDP configuration.

The last reported problem has been that the SP did not receive any attributes -- and the error message described this well. Do whatever is necessary to include attributes and if the login does not succeed, look for error messages. With a bit luck, those will again give precious hints.

1 Like

It seems that the Zulip-based SP is not requesting any attributes, despite being configured to do so. I'm investigating further.
Based on my reading of the dm.zope.saml2 code, until the Zulip-based SP actually sends the "wanted attributes", the Plone-based IdP will not return any attributes.

I've brought this up over at Zulip chat https://chat.zulip.org/#narrow/stream/31-production-help/topic/SAML.20metadata (requires a login).

This is indeed the case.

A workaround could be to fetch the metadata indirectly and add the missing information for requested attributes.

I'm clear now that attributes are declared in the SP's metadata.
My expectation from the Zulip-based SP is that the metadata would include an AttributeConsumingService entry containing the RequestedAttribute entries.

In this example (found here: SAML metadata - Wikipedia) the AttributeConsumingService declares each RequestedAttribute inline.

 <md:EntityDescriptor entityID="https://sso.example.com/portal" validUntil="2017-08-30T19:10:29Z"
…”>
    <!-- insert ds:Signature element (omitted) -->
…
    <md:SPSSODescriptor WantAssertionsSigned="true" protocolSupportEnumeration="urn:oasis:names:tc:SAML:2.0:protocol">
…
      <md:NameIDFormat>urn:oasis:names:tc:SAML:2.0:nameid-format:transient</md:NameIDFormat>
      <md:AssertionConsumerService index="0" Binding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST"  Location="https://service.example.com/SAML2/SSO/POST"/>
      <md:AttributeConsumingService index="0">
        <md:ServiceName xml:lang="en">Example.com Employee Portal</md:ServiceName>
        <md:RequestedAttribute isRequired="true"
          NameFormat="urn:oasis:names:tc:SAML:2.0:attrname-format:uri"
          Name="urn:oid:1.3.6.1.4.1.5923.1.1.1.13" FriendlyName="eduPersonUniqueId"/>
        <md:RequestedAttribute
          NameFormat="urn:oasis:names:tc:SAML:2.0:attrname-format:uri"
          Name="urn:oid:0.9.2342.19200300.100.1.3" FriendlyName="mail"/>
      </md:AttributeConsumingService>
    </md:SPSSODescriptor>
…
  </md:EntityDescriptor>

By contrast, my SP metadata does NOT contain a AttributeConsumingService entry at all :grimacing:.

<?xml version="1.0" encoding="UTF-8"?>
<md:EntityDescriptor xmlns:md="urn:oasis:names:tc:SAML:2.0:metadata" cacheDuration="P10D" entityID="https://zulipsp.xxxxxxxxxx.com">
   <md:SPSSODescriptor AuthnRequestsSigned="false" WantAssertionsSigned="false" protocolSupportEnumeration="urn:oasis:names:tc:SAML:2.0:protocol">
      <md:KeyDescriptor use="signing">
         <ds:KeyInfo xmlns:ds="http://www.w3.org/2000/09/xmldsig#">
            <ds:X509Data>
               <ds:X509Certificate>MIICdDCCAd2gAw...evUJxm1Y/dl+lQbY=</ds:X509Certificate>
            </ds:X509Data>
         </ds:KeyInfo>
      </md:KeyDescriptor>
      <md:NameIDFormat>urn:oasis:names:tc:SAML:1.1:nameid-format:unspecified</md:NameIDFormat>
      <md:AssertionConsumerService Binding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST" Location="https://zulipsp.xxxxxxxxxx.com/complete/saml/" index="1" />
   </md:SPSSODescriptor>
   <md:Organization>
      <md:OrganizationName xml:lang="en-US">zulipsp</md:OrganizationName>
      <md:OrganizationDisplayName xml:lang="en-US">xxxxxxxxxx zulipsp</md:OrganizationDisplayName>
      <md:OrganizationURL xml:lang="en-US">https://zulipsp.xxxxxxxxxx.com</md:OrganizationURL>
   </md:Organization>
   <md:ContactPerson contactType="technical">
      <md:GivenName>Technical team</md:GivenName>
      <md:EmailAddress>david.bain@xxxxxxxxxx.com</md:EmailAddress>
   </md:ContactPerson>
   <md:ContactPerson contactType="support">
      <md:GivenName>Support team</md:GivenName>
      <md:EmailAddress>david.bain@xxxxxxxxxx.com</md:EmailAddress>
   </md:ContactPerson>
</md:EntityDescriptor>

Also... I took a look at the dm.zope.saml2 code and it clearly looks for a AttributeConsumingService (which it names acs) then iterates over all the RequestedAttribute entries. Further confirmation that I need to "massage" Zulip-based SP into placing the RequestedAttribute entries in the metadata.

It looks like Zulip as an SP does not send RequestedAttributes.
So I will need to "force" Plone to send the attributes.

I got it to work :tada: . Horrible workaround. I went into the code and "forced" it to return the attributes without a RequestedAttribute from the Zulip SP.

@dieter thanks for your feedback and guidance.

Now to figure out if this needs to be a pull request or a monkey patch :thinking:.