Possible problem with Smart Connect login using oauth2

[jritchey_jaxcon]

I'm using the swagger UI to view and test REST api (planning on writing a client to utilize it).

During testing, it seems that the oauth2 token will expire in a relatively short period, and then you must login again.

This would be fine, but attempting to login a second time often fails, with an error message:

"The User Name or Password were incorrect"

I thought it was just a problem with the UI, but in checking the chrome console, I'm seeing the correct password is in fact sent, indicating a problem with the MongooseEwsRestProvider endpoint

I've blurred it out, but I can assure you the correct password is in the form data.

Additionally, I've noticed that the same oauth token is given with each unique session.  (I don't know how oauth2 works, but at least in basic/digest auth, the token is usually unique per session, so perhaps this is related to the problem I'm seeing).

EDIT: on closer observation, the token is unique per session, they just have a lot of characters in common at the beginning.

[sesa180908_brid]

A couple of things first.

• From the looks of your captures, you are probably using the SoapEwsRestProvider (used for SBO etc) class not the MongooseEwsRestProvider (which is for a native SmartConnector EWS Servers).  Regardless, the OAuth interaction with Swagger would be identical.
• The time the Bearer Token is valid is entirely within your control.  By default it is 5 minutes but you can change the value in the Endpoint Configuration (see Http Configuration => Access Token Time Span Minutes).
• At the present time, SmartConnector doesn't utilize OAuth refresh tokens although you could always request a new token at any time.

I've tested what you are reporting and cannot replicate the issue.  My test was as follows (I was also using Chrome):

1. Configure a SoapEwsRestProvider backed by SBO (both running locally in my test bed).
2. Set the configuration Access Token Expire Time Span Minutes to 1 minute.
3. Start the Endpoint
4. Open Swagger
5. Authenticate with valid credentials.  Observe acceptances of credentials.
6. Execute GetRoot.  Observe valid response.
7. Wait until token expires
8. Execute GetRoot.  Observe "Message": "Authorization has been denied for this request." response.
9. Click Login again.  Observe page reload.
10. Execute GetRoot.  Observe valid response.

I've done the above with Fiddler open and not and with the Chrome debugger open and not.

Perhaps there is something else going on here.

Are you connecting to an an ES or an AS?

What version?

Where is SBO in relation to SmartConnector (i.e. is it co-located on the same physical machine)?

Are there any other proxy servers at play?

Can you provide a screen shot of your Endpoint Configuration (please expand the Http Configuration node)?

[jritchey_jaxcon]

You're right, it's SoapEwsRestProvider, not Mongoose:

Here is my Provider config and details:

and I'm connecting it to an ES on the same host:

The problem has been intermittent for me, but it happens often enough that it could be of some concern.

my steps are pretty much the same as yours:

1) login with username and password

2) send a request for GET /Root

3) send a request for GET /Containers/{Root.id}/Children

4) send a request for GET /Values/{Test.id}

5) wait for token to expire

6) attempt another GET /Values/{Test.id} (receives expected 401 response)

7) click the login button (sometimes it works, and sometimes i get an alert stating wrong username and password)

[JeffBowman]

Hi Josh,

Is there any way you could turn on SmartConnector logging on to maybe see if we can see what is happening here?

In Service Settings set the 'Logging Level' to TRACE and in the Logging Filters turn on 'Ews Consume' and 'Rest Serve'.

If you can get us a log file with these settings on and this happening, this might shed some light onto what is happening.

Regards,

-Jeff

[jritchey_jaxcon]

I'm willing to give that a try.

[jritchey_jaxcon]

okay, I've enabled TRACE logging and Consume/Serve filters, but I'm not sure where to retrieve the logs.

[sesa180908_brid]

Logs are stored in C:\ProgramData\SmartConnector\Logs

[sesa180908_brid]

You can email them to me.

[jritchey_jaxcon]

alright, I sent you the log file.  I was able to replicate the problem during the last few minutes of logging.

I also observed something I haven't seen up until now.  it may be a separate issue:

when I requested /Root after the expiration (via the 'Try it out!' button), instead of the expected 401 Unauthorized, I received a 500 Internal Server Error, with data:

{"Message":"The underlying connection was closed: The connection was closed unexpectedly."}

a second try got the expected 401 response.

after that, I clicked login, and got the 400 (bad request), with data:

{"error":"invalid_grant","error_description":"The User Name or Password were incorrect."}

[sesa180908_brid]

I see that in the logs but I'm not seeing any Rest Serve entries.  I think you turned on EWS Serve instead because I am seeing calls to the SmartConnectorWeatherService sample server.

Can you turn OFF Ews Serve and turn ON Rest Serve and resend me the logs.

[jritchey_jaxcon]

Looks like you're right... Sorry about that.  I'll email you the correct logs in a moment.

[jritchey_jaxcon]

I'm having trouble replicating the problem now that Rest Serve logging is enabled.  I'm going to try turning it back off and see if it increases the chances of seeing the problem again.  Very strange...

[jritchey_jaxcon]

Unfortunately, I can't seem to consistently replicate the problem anymore.  I have been able to test my REST client against it, and it seems to be working for the time being...

I've reenabled the logging for Rest Serve,
and I'll report back with those logs if the problem arises again.

[sesa180908_brid]

For future reference, what version of SBO are you using?

[jritchey_jaxcon]

SBO 1.8.1.79

[jritchey_jaxcon]

I may have replicated the problem (or similar to it) this morning.  I sent you another email containing the logs.  It should contain both Ews Consume and Rest Serve logs, and is now excluding Ews Serve logging.

Hopefully it helps identify the problem.

[sesa180908_brid]

The only thing that the log files show is that the server (SBO) is terminating the connection.  This can occur if SBO has exhausted it's pool of available client connections. 

Do you have other EWS clients which are hitting the same endpoint?

[jritchey_jaxcon]

Should I be looking for too many clients attempting to connect to the ES,

or too many clients attempting to connect to the SmartConnector/REST endpoint?

[jritchey_jaxcon]

This is my best approximation of our network configuration right now.  I don't see anything that would obviously be consuming all of the ES or REST endpoint connections at present, and wireshark capture on ports 8082, 8083, and 808 isn't showing a lot of activity.

[sesa180908_brid]

This issue is similar in symptoms to a known issue which is documented in the SmartConnector release notes (See Bug #348).  The only difference is that the issue has only been reported when the communication between SmartConnector and SBO is over TLS.

In order to determine if this issue is the similar, we will need a Wire Shark capture which demonstrates the issue.

Also, from your diagram the EWS server which is supplying data to the REST endpoint (local 1.8 ES on 808) is also serving EWS to a remote client (.52 1.8 ES) and consuming EWS (local 1.7 ES on 807) and SC Weather project (localhost 5300).  You should consider disabling this other communication until we determine where the root cause is.

[jritchey_jaxcon]

Okay, I'll go ahead and disable those other communications, and I will attempt to capture the problem in wireshark, as well as a an example of a successful communication.

[jritchey_jaxcon]

alright, I've emailed the wireshark capture.  contains several failed attempts to POST /GetToken, and finally a successful attempt.

[jritchey_jaxcon]

The capture looks characteristic of a similar problem I was observing earlier, when attempting to login directly to Ews via Soap UI:

Problems with EcoStruxureware on loopback​

For me, the problem only occurs on loopback, and the wireshark capture shows a 'Connection Reset' response immediately following the client's attempt to respond to the digest challenge with username and password.

Hopefully this additional information of of use

[sesa180908_brid]

Josh,

From analyzing the captures, the symptoms you are experiencing are very similar to the those that had formerly only been reported when SmartConnector consumed EWS from SBO over TLS (HTTPs).  We believe the root cause is due to SBO issuing a TCP RST immediately after the expected 401 response during the HTTP Digest authentication handshake.  This behavior, while technically acceptable under RFC 7230, is known to potentially cause issues for clients. 

We've updated the EwsClient class in SmartConnector so that it will be more tolerant of this condition.  This update is now available as a beta (v2.2.125) from SmartConnector Server.  You should have access to this beta download now if you wish to try this version. 

[jritchey_jaxcon]

thanks! glad to hear that.  I will checkout the beta as soon as I can get around to it.

[sesa180908_brid]

Josh,

v2.2.127 has been posted to smartconnectorserver.  This version contains the patch that will address the issue you experienced.