Difference between revisions of "My NextServiceMgr"

The Unofficial SmartThings Blog
Jump to navigation Jump to search
(Issue #14: My Next devices are not updating or my auth tokens are lost)
(Issue #14: My Next devices are not updating or my auth tokens are lost)
Line 412: Line 412:
  
 
* Please take note that, during the steps described below, you need to log in to home.nest.com in order to get the right login info!!   
 
* Please take note that, during the steps described below, you need to log in to home.nest.com in order to get the right login info!!   
 +
 +
* Execute ''MyNextManager'' again to re-authorize the smartapp.
  
  

Revision as of 11:19, 18 January 2021



Category: My Apps

Author:Yves Racine

ST Community handle: http://community.smartthings.com/users/yvesracine/activity Hubitat Community handle: https://community.hubitat.com/u/yracine66/activity


Company Link: http://www.ecomatiqhomes.com#!home/mainPage



If you like MyNextServiceMgr and related Next devices, please support the developer by clicking on the paypal link below:

https://paypal.me/ecomatiqhomes



Summary

MyNextServiceMgr: the smartapp is the Service Manager for all Next devices (Tstat, Protects, Cams): it instantiates the Nest device(s) and polls them on a regular basis"

Supported Features

  • Multiple devices can be selected to be exposed to SmartThings or Hubitat.
  • Do a poll at regular intervals (input parameter)
  • Notify you if there is any issue with the ST/Nest connection (optional).
  • Can send summary reports via Ask Alexa and/or nofications about your Next devices' usage (weekly, daily).

Prerequisites

  • Nest devices fully operational and connected via wi-fi to Nest
 Please note that all Nest devices are supported except the Nest thermostat E in the UK (with the heat link) and the Nest Secure Alarm. The Nest 
 Hello only appears as a Nest Cam (not other special features).


  • SmartThings or Hubitat hub fully operational
  • Some basic skills to add custom DTH and smartapps (see troubleshooting section, item 1)
  • Nest setup completed for your Nest devices under your Nest Primary account (don't use any Nest secondary accounts for the integation with your hub!!).
  • Determine your shard, please consult this thread:

https://community.smartthings.com/t/faq-how-to-find-out-what-shard-cloud-slice-ide-url-your-account-location-is-on/53923

Or the SmartThings documentation here for more details:

http://docs.smartthings.com/en/latest/publishing/index.html#ensure-proper-location

UI and integration setup workflow

MyNextManagerP1.pngMyNextManagerStruct.jpg MyNextManagerP6.pngMyNextManagerP7.png MyNextManagerV2 sensors.jpgMyNextManagerP9.png

MyNextManagerP10.png MyNextManagerP11.pngMyNextManagerP12.png MyNextManagerP13.png

Code location: https://github.com/yracine/device-type-myNext/tree/master/smartapps

Instructions for installation (readme): https://github.com/yracine/device-type-myNext/blob/master/README.md

Troubleshooting

See troubleshooting section under the different Next devices


.................................................................................................................................

Issue #1: I don't know how to create a custom smartapp

.................................................................................................................................


Hubitat

Copy the code from the text files under apps in the Hubitat IDE, and then press "Save".

HubitatSmartapp page1.PNG

Enable "oAuth" for the smartapp.

HubitatSmartapp OAuth.png


SmartThings

See the explanations of a fellow community member here:

http://community.smartthings.com/t/faq-an-overview-of-using-custom-code-in-smartthings/16772

http://thingsthataresmart.wiki/index.php?title=Using_Custom_Code

And, here are some basic explanations with screenshots.

To avoid any issue, please to do the steps below using a well known browser (Chrome, IE, Edge) from a desktop/laptop (not a tablet) to copy and paste the code into the IDE.

N.B. To create a DTH (Device Type Handler), it's basically the same steps above, but you need to click on My Device Handlers in the upper menu of the IDE, copy and paste the code, hit the create button and after, save & publish.

  • a) Point your browser to you SmartThings IDE for your country (i.e. http://ide.smartthings.com or whatever your shard is, refer to the prerequisites at the readme) and Log In.

Loginscreen.jpg

  • b) Once you are logged in, find the My SmartApps link on the top of the page. Clicking My SmartApps will allow you to produce a new SmartApp.

MySmartApps.png

  • c) Find the button on this page labeled +New SmartApp and click it.

+NewSmartApp.png

  • d) Since you already have the code in your computer’s clipboard, find the tab along the top section called From Code. In the area provided, paste (typically CTRL+V) the code you copied from GitHub. Click Create in the bottom left corner of the page.

NewSmartAppCreate.png

  • e) This will bring up another page, with the code now formatted within the IDE. If the code was copied correctly, there are no other steps except to save and publish the code. In the upper right corner of the page, find and click Save. Now, click Publish (For Me), and you should receive a confirmation that the code has been published successfully.

SavePublish.png

  • f) To enable OAuth, first find and click the App Settings button in the upper right corner of the page.

To access the App Settings section

MyNextManagerV2 AppSettings.png

From here, find the OAuth section toward the bottom of the page.


OAuthBtn.png

  • g) Clicking the OAuth link will reveal a button labeled Enable OAuth in Smart App. Click this button. The screen will change, giving you a unique code for your Client ID and Client Secret. These are the foundations of the security of your app and should be kept secret. You do not need to memorize or write down these codes; nor do you need to add any other information to this page. OAuth simply needs to be enabled for

OAuth.jpg


For the steps required for the login info, please refer to the screenshots below:

https://thingsthataresmart.wiki/index.php?title=My_NextServiceMgr#Issue_.2314:_My_Next_devices_are_not_updating_or_my_auth_tokens_are_lost

.................................................................................................................................

Issue #2: I created the smartapp in the IDE, but I don't know where to execute it

.................................................................................................................................

For the initial setup, under the new Samsung connect app, click on '+' sign in the upper right corner of the main page, and then on Smartapp. The smartapp should be under the Custom section.

If you still do not see your smartapp, please make sure that you have published it in the IDE!


For any subsequent execution, under the Samsung connect app, click on the 'hamburger' menu in the upper left corner. The list of installed smartapps will then appear under Smartapps.


P.S. If you don't see your smartapp, it could be that you're not using the right shard for the installation. Refer to

https://community.smartthings.com/t/faq-how-to-find-out-what-shard-cloud-slice-ide-url-your-account-location-is-on/53923

Or the SmartThings documentation here:

http://docs.smartthings.com/en/latest/publishing/index.html#ensure-proper-location


.................................................................................................................................

Issue #3: java.lang.NullPointerException: Cannot get property 'X' on null object

.................................................................................................................................


Just update your smartapp by going through all the smartapp setup pages by pressing 'Next' till 'Done'.

The smartapp was recently modified to include some defensive code to ensure that the smartapp is rescheduled. This is intended to alleviate some of the ST scheduling issue as discussed in these threads:


http://community.smartthings.com/t/scheduler-and-polling-quits-after-some-minutes-hours-or-days/16997

http://community.smartthings.com/t/schedule-not-working/3895

http://community.smartthings.com/t/is-it-me-or-schedules-not-firing-this-evening/10176


.................................................................................................................................

Issue #4: java.util.concurrent.TimeoutException: Execution time exceeded 20 app execution seconds

.................................................................................................................................


ST has introduced some generic rate limiting rules for all smartapps and device handlers. For cloud-to-cloud integrations, these rate limiting constraints are sometimes too low, and this exception can be raised from time to time.

There are some threads about it in the ST community forum:

http://community.smartthings.com/t/rate-limiting-too-restrictive-max-execution-time-exceptions/17985

http://community.smartthings.com/t/execution-times-increasing/19979

http://community.smartthings.com/t/unschedule-api-execution-time-too-long/11232

.................................................................................................................................

Issue #5: I want to upgrade the code but I don't know how

.................................................................................................................................

To get the new minor releases, you just need to use the original sellfy download link that was sent to you when you made your contribution.

Most of the time (unless specified otherwise), you just need to copy and paste the smartapp's code from the zip file.

Click on the smartapp that you want to update in the IDE. Go to

https://graph.api.smartthings.com/ide/apps (or whatever your shard is).

Paste the code into the code window (and make sure to copy all the lines)

And save & publish.

The same principle applies for the DTHs (Device Type Handlers).


.................................................................................................................................

Issue #6 - My new Ask Alexa's Message Queues are not showing up in the smartapp

.................................................................................................................................


You'd need to execute the MyNextServiceMgr smartapp under Automation/Smartapps in the ST mobile app by doing "Next" till "Done" to subscribe to the new AskAlexaMQHandler. The new handler will then be able to process any Queue refresh value from Ask Alexa.


You can then go to the AskAlexa smartapp, and make any queue changes... The updated queue values will be then sent to the smartapp.

................................................................................................................................

Issue #7: I want to be notified of any exceptions coming from ST and/or Nest

.................................................................................................................................

If you want to be notified of any serious ST/Nest exceptions, you can set the following flag to true in MyNextServiceMgr(in Other Settings -last page):


   section("Handle/Notify any exception proactively [default=false, you will not receive any exception notification]") {
      "Handle exceptions proactively?"
   }


The smartapp will then send you notifications when exceptions occur.


.............................................................................................................................

Issue #8: There is a groovyx.net.http.HttpResponseException: Too Many Requests exception in my logs

.............................................................................................................................

There are also variants of this message for devices: "(...) Nest Throtting in progress (...)" in the list of Events or the logs (IDE).

There could be different reasons why you have this exception thrown by Nest.

  • 1) Not using the caching in the smartapp

If you disable the following option in the smartapp:

   section("To refresh your current structures, don't use the cache [default=cache is used for better performances") {	
       "use of cached structures including devices?"
   }        

This means that every request to Nest will be sent without using any caching. This could be an issue as indicated in 2).

  • 2) Polling interval is too agressive for your use cases

If your polling interval is 1-5 minutes, you may encounter this exception as Nest can throttle your requests when too many are sent in a given timeframe. Nest APIs have 1 call per minute and 10 PUTs per hour limitations.

You may then need to increase the polling interval to 5-10 minutes, especially if you have many Nest devices under your account.

Refer to the Nest doucmentation about the Nest rate limiting:

https://developers.nest.com/guides/api/data-rate-limits




.............................................................................................................................

Issue #9: I'd like to uninstall My Next devices

.................................................................................................................................


a) Like any connected device, first you'd need to unselect the next devices and remote sensors device(s) from all smartapps and/or routines (ex. Amazon echo).

To know which smartapps are using My Next devices, go to

https://graph.api.smartthings.com/device/list (or whathever your shard is)

b) Click on each of My Next devices instantiated under your ST account

And note all smartapps under the list "In Use By".

c) Then, you'd need to unselect the device(s) from the smartapps or routines using the ST mobile app.

d) Finally, you can remove the installed MyNextManagerV2 smartapp by pressing on the corresponding 'delete' button after clicking on the smartapp name and the ':' sign in the upper right corner.

To access the installed smartapps, go first under the hamburger menu in the upper left corner of the Samsung connect app, and press on 'Smartapps' to get the list of installed smartapps under your ST account.

If you have any delete issues, you can also delete the devices directly from the IDE by clicking on the device, pressing the 'gear button (upper right), and then remove at the bottom.


.............................................................................................................................

Issue #10: I want detailed Logging/Notifications or I want to enable Ask Alexa Notifications

......................................................................................................................................

Hubitat

  • a) go to your installed apps and select the right app amongst the list

go to http://192.168.xx.xx/installedapp/list (insert your own hub's IP address)

  • b) In Notifications & Other Settings Setup> On the last setup page, set the following flag to true
   "Detailed Logging & Notifications?"


  • c) go to the logs section of your hub

http://192.168.xx.xx/logs (insert your own hub's IP address)

SmartThings


In order to have detailed Logging/Notifications, you'd need to set the following parameters in the smartapp:

  • a) Notifications & Other Settings Setup> On the last setup page, set the following flag to true
   "Detailed Logging & Notifications?"


  • b) To receive push notifications, answer 'Yes' to the following input parameter
   "Send a push notification?", metadata: [values: ["Yes", "No"]]


  • c) To receive the notifications configured above through Ask Alexa, set the following flag to true
   "Ask Alexa verbal Notifications?"


N.B. The logs are available under the IDE at:

https://graph.api.smartthings.com/ide/logs (or whatever your shard is, under Live Logging in the IDE)

You can click at the top of the window on the smartapp name to filter the logs.


...........................................................................................................................

Issue #12: Some of my devices under the new Samsung app are disconnected

.................................................................................................................................


If some of your Next devices are disconnected under the new Samsung app, you may have created many of the same devices by executing MyNextManager many times... Under the new Samsung mobile app, you then need to clean them up by pressing the "edit" button next to your location and removing them.


...............................................................................................................................


Issue #14: My Next devices are not updating or my auth tokens are lost

.................................................................................................................................


For your first setup or if some of your Next devices are not updating because you may have lost your auth tokens, please follow the steps below.


  • MAKE SURE TO get the right login info (a or b) as explained below, because if you are
   a) Nest Account user or 
   b) Google Account User (migrated Nest account to Google)
   the instructions set ARE NOT THE SAME
  • Log off first from any Nest active session
  • Start the Chrome browser in incognito mode or clear your cache first.
  • Please take note that, during the steps described below, you need to log in to home.nest.com in order to get the right login info!!
  • Execute MyNextManager again to re-authorize the smartapp.



  1. a) LOGIN INFO REQUIRED FOR NEST ACCOUNT Users (NOT migrated to a Google account)

Notes:

  • Once you log in to your Nest account as described in the steps below, please keep the connection active to avoid any disconnect (do not log off, but you can close your browser and even turn off your desktop/mobile), and don't change your Nest account password or 2FA settings.
  • If you lose your auth tokens, then you'd need to redo the steps below and copy over the new nest access_token and user id back to the IDE. You can then reset the Nest connection by executing MyNextManagerV2 under the installed smartapps (under the hamburger menu in the upper left corner in the Samsung connect app) by pressing "Next" till "Save").
  • You don't need to re-install the devices, just reset the Nest connection.
  • Google recently introduced reCAPTCHA when logging to Nest. That means username and password cannot be used directly any more. Instead, you have to obtain `user_id ` and `access_token` for your account by logging in manually.


__________________________________________________________________


7 Steps (for nest users not migrated to a Google Account):

__________________________________________________________________


1. [Chrome] To do that, open `developer tools` in your Chrome browser, switch to the `Network` tab,

2. [Chrome] Hit preserve Logs in the Network tab

See screenshot below

HomeNestLogindeveloperMode.png

3. [Chrome] Log in to home.nest.com and look for the request similar to https://home.nest.com/session?_=157XXXXXX.

4. [Chrome] In the Network tab, you can use the filter "session" to get the exact info needed.

See screenshot below


HomeNestNetworkWindow.png

It may be helpful to unfold the data in the preview to find exactly the right fields.


Screenshot showing session info variables

5. [Chrome] You will find `userid` and `access_token` in the response to the request. Use the access_token from the 2fa_state section of the session variable.


6. [Chrome->ST] MyNestManagerV2- header section of the code: Copy over the Nest login information to the corresponding nest_* fields in the code (ST IDE) for MyNextManagerV2, refer to step 2g) below at https://github.com/yracine/device-type-myNext/blob/master/README.md#2-create-a-new-smartapp-mynextmanagerv2.

In the header section (code) of MyNextManagerV2, you will see "INSERT THE NEST LOGIN INFORMATION BELOW".

FYI,the login info for Nest account users looks like the following (your Nest user id will be different):

nest_user_id=4783742 nest_access_token=b.4783742.xXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

7. [ST IDE] MyNestManagerV2- header (code) section: After copying and pasting the login info, press "save" and "publish" to save/publish the code with your Nest login variables. Don't insert anything in the google section.

Do not copy the double quotes in the global variables.

See screenshot from the code section (header) in the IDE below

NextManagerVariables.PNG





  1. b) LOGIN INFO REQUIRED FOR GOOGLE ACCOUNT USERS (MIGRATED NEST USERS)


Notes:

  • You need to start a new Google session. Once you log in to your Google account as described in the steps below, please keep the connection active to avoid any disconnect (do not log off, but you can close your browser and even turn off your desktop/mobile), and don't change your Google account password or 2FA settings.
  • If you lose your auth tokens, then you'd need to redo the steps below and copy over the new google cookie and issue_token_url back to the IDE. You can then reset the Nest connection by by executing MyNextManagerV2 under the installed smartapps (under the hamburger menu in the upper left corner in the Samsung connect app) by pressing "Next" till "Save").


  • You don't need to re-install the devices, just reset the Nest connection.



__________________________________________________________________


14 Steps (for nest users who have migrated to a Google Account):

__________________________________________________________________

The values of `issue_token` and `cookie` are specific to your Google Account. To get them, follow these steps (only needs to be done once, as long as you stay logged into your Google Account).

1. [Chrome] Open a Chrome browser tab in Incognito Mode (or clear your cache).

2. [Chrome] Open Developer Tools (View/Developer/Developer Tools).


To access the developer mode in the Chrome Browser at the home.nest.com login page

See screenshot below

HomeNestLogindeveloperMode.png

3. [Chrome] Click on 'Network' tab.

4. [Chrome] Make sure 'Preserve Log' is checked in the Network tab.

5. [Chrome] In the 'Filter' box, enter `issueToken`

6. [Chrome] Go to `home.nest.com`, and click 'Sign in with Google'. Log into your account (i.e., start a new session).

7. [Chrome] One network call (beginning with `iframerpc`) will appear in the Dev Tools window. Click on it.

See screenshot below

Home google login issue token.PNG


9. [Chrome] In the 'Filter' box, enter `oauth2/iframe` to get login information from Google/Nest under the Network window

10. [Chrome] Several network calls will appear in the Dev Tools window. Click on the last `iframe` call.

11.[Chrome] In the Headers tab, under Request Headers, copy the entire `cookie` (usually it starts with 'OCAK=...' or 'SID=...' or 'SMSV=...', but it can start with other values as well - **include the whole string which is several lines long and has many field/value pairs** - do not include the `cookie:` name). You can copy it in your preferred editor to split it in multiple fields of similar length.

See screenshot below

Home google login cookie.PNG


12. [Chrome->ST IDE] MyNestManagerV2- header section of the code. This google "cookie" needs to be copied over in the code (header) section of the MyNextManagerV2 smartapp under the instructions saying 'INSERT GOOGLE LOGIN INFO BELOW'. See https://github.com/yracine/device-type-myNext/blob/master/README.md#2-create-a-new-smartapp-mynextmanagerv2.

13. [Chrome->ST IDE] MyNestManagerV2- Make sure that the whole cookie from Google is contained in google_cookiep1. There is no need to split the cookie anymore (the other google_cookie p2-p6 variables are used for backward compatibility. Please make sure to avoid inserting extra spaces or any other characters when you copy the google cookie as Google will not accept it and the list of devices will be empty. Don't insert anything in the Nest section.

14. [ST IDE] MyNestManagerV2- After copying and pasting the login info, press "save" and "publish" at the right corner of the IDE to save/publish the code with your variables.

For Google account users, please make sure of the following when doing the "copy & paste":

  • Please make sure to avoid inserting extra spaces or any other characters when you split the cookie as this will not be accepted by Google, and the cookie will not be recognized (and your list of devices will be empty).

See screenshot from the code section (header) in the IDE below

NestManagerGoogleVariables.PNG


...............................................................................................................................

Issue #15: My Nest Tstat doesn't show up in MyNextManagerV2's list of devices

.................................................................................................................................


After some research, I found a potential workaround for some Nest tstats that don't show up in MyNextManagerV2. This is not related to my code per se, but the way your tstat is linked to your Nest account.

  • Go to settings on the Nest Thermostat (physical) unit that isn't showing up
  • Scroll to the Reset Item
  • Select the WWN (Works With Nest) reset option
  • Reset the device and re-add it to your Nest app


Try running MyNextManagerV2 again under Automation/smartapps in the ST classic app (with the same credentials).