MQTT Username and P...
 
Notifications
Clear all

[Solved] MQTT Username and Password

Page 1 / 2

stevecos
(@stevecos)
Active Member
Joined: 1 month ago
Posts: 13
Topic starter  

I am adding an MQTT input to Mycodo (specifically, data from Cayenne).  The interface makes it very easy to see where to put the hostname, port, Client ID, and Channel ('Name'), but I don't see where to put my username and password.  Can you please point me in the right direction? 
Thanks.  Steve


Quote
Topic Tags
Kyle Gabriel
(@kylegabriel)
Member Admin
Joined: 6 years ago
Posts: 570
 

I just added the option in commit e886c14. If you would like to test, you will need to upgrade to master. Please let me know if it works if you do test. Thanks.

Mycodo Developer


ReplyQuote
stevecos
(@stevecos)
Active Member
Joined: 1 month ago
Posts: 13
Topic starter  

Success!! ... in only that way that a developer can understand.  Otherwise known as a triumphant failure.

  • Getting the right values in place to retrieve data from Cayenne took some mental gymnastics.  Will document that once I have it working.  Could you please point me to a page where I can create some suggested documentation?  (Which will give me an idea of an appropriate format.)
  • Evidence (see log below) suggests I managed to retrieve some data, but it arrived as a string that could not be turned into a float.  The data should have been '5289.0'
  • For your interest, this code is an example of how I retrieve data from Cayenne very successfully.

Will keep fiddling and add an update if I get further.

2021-01-31 06:06:28,984 - DEBUG - mycodo.inputs.mqtt_paho_68ff4a34 - Client created with ID 8bd9d700-11eb-8779-7d56e82df461
2021-01-31 06:06:28,985 - DEBUG - mycodo.inputs.mqtt_paho_68ff4a34 - Sending username and password credentials
2021-01-31 06:06:28,985 - DEBUG - mycodo.inputs.mqtt_paho_68ff4a34 - Connecting MQTT callback functions
2021-01-31 06:06:28,985 - DEBUG - mycodo.inputs.mqtt_paho_68ff4a34 - MQTT callback functions connected
2021-01-31 06:06:29,325 - INFO - mycodo.inputs.mqtt_paho_68ff4a34 - Connected to mqtt.mydevices.com as 8bd9d700-11eb-8779-7d56e82df461
2021-01-31 06:06:29,326 - DEBUG - mycodo.inputs.mqtt_paho_68ff4a34 - Subscribed to MQTT channel 'v1/6375a470-cff9-11e7-86d0-83752e057225/things/8bd9d700-3131-11eb-8779-7d56e82df461/data/26'
2021-01-31 06:06:29,329 - INFO - mycodo.controllers.controller_input_68ff4a34 - Activated in 624.5 ms
2021-01-31 06:06:29,558 - DEBUG - mycodo.inputs.mqtt_paho_68ff4a34 - Connected to 'None', rc: 0
2021-01-31 06:06:29,765 - DEBUG - mycodo.inputs.mqtt_paho_68ff4a34 - Subscribing to mqtt topic: None, 1, (0,)
2021-01-31 06:08:29,843 - DEBUG - mycodo.inputs.mqtt_paho_68ff4a34 - Message received: Channel: v1/6375a470-cff9-11e7-86d0-83752e057225/things/8bd9d700-3131-11eb-8779-7d56e82df461/data/26, Value: analog_sensor,null=5289.0
2021-01-31 06:08:29,844 - ERROR - mycodo.inputs.mqtt_paho_68ff4a34 - Message doesn't represent a float value.
Traceback (most recent call last):
  File "/home/pi/Mycodo/mycodo/inputs/mqtt_paho.py", line 230, in on_message
    value = float(msg.payload.decode())
ValueError: could not convert string to float: 'analog_sensor,null=5289.0'

ReplyQuote
stevecos
(@stevecos)
Active Member
Joined: 1 month ago
Posts: 13
Topic starter  

Oops!  Just looked at the log properly ... of course "analog_sensor,null=5289.0" is never going to be a float!  Might require bulk thinking there. 樂


ReplyQuote
stevecos
(@stevecos)
Active Member
Joined: 1 month ago
Posts: 13
Topic starter  

I've got it going OK!

Based on my own code linked above, I added an extra line into the on_message function, line 215, of /home/pi/Mycodo/mycodo/inputs/mqtt_paho.py

def on_message(self, client, userdata, msg):
  datetime_utc = datetime.datetime.utcnow()
  self.logger.debug("Message received: Channel: {}, Value: {}".format(
  msg.topic, msg.payload.decode()))
  measurement = {}
  channel = None
  for each_channel in self.channels_measurement:
    if self.channels_measurement[each_channel].name == msg.topic:
    channel = each_channel

    if channel is None:
      self.logger.error("Could not determine channel for '{}'".format(msg.topic))
     return

  try:
    null,Data = str(msg.payload.decode()).split(sep='=')
    value = float(Data)
  except Exception:
    self.logger.exception("Message doesn't represent a float value.")
  return
...

This change might work well for Cayenne, but you will need to think about how to make a more generalised solution!

Apparently my battery voltage is down to 5.236 volts now (not surprising - it is night here in New Zealand).  At least I can be confident that data will stay for as long as I want it, unlike a Cayenne free account which (quite understandably) starts to consolidate data, dropping intermediate values, after a day.

My sensor only records values once every hour, so testing will take a while, but I'll toss a few more channels in and see how that goes.

Thanks for all the work you've put into this system.  It is a privilege to be able to contribute a small part.

Steve


ReplyQuote
stevecos
(@stevecos)
Active Member
Joined: 1 month ago
Posts: 13
Topic starter  

I went to Data->Data to add another data point, but that through an exception.  Rebooted the Pi it is running on and now it works.  If it happens again I'll log an issue on GitHub.  Here is the Traceback:

Error (Full Traceback):

Traceback (most recent call last):
  File "/home/pi/Mycodo/env/lib/python3.7/site-packages/flask/app.py", line 2447, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/pi/Mycodo/env/lib/python3.7/site-packages/flask/app.py", line 1952, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/home/pi/Mycodo/env/lib/python3.7/site-packages/flask_restx/api.py", line 639, in error_router
    return original_handler(e)
  File "/home/pi/Mycodo/env/lib/python3.7/site-packages/flask/app.py", line 1821, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/home/pi/Mycodo/env/lib/python3.7/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/home/pi/Mycodo/env/lib/python3.7/site-packages/flask/app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/pi/Mycodo/env/lib/python3.7/site-packages/flask/app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/home/pi/Mycodo/env/lib/python3.7/site-packages/flask_login/utils.py", line 272, in decorated_view
    return func(*args, **kwargs)
  File "/home/pi/Mycodo/mycodo/mycodo_flask/routes_page.py", line 1837, in page_data
    dict_inputs = parse_input_information()
  File "/home/pi/Mycodo/mycodo/utils/inputs.py", line 109, in parse_input_information
    input_custom = load_module_from_file(full_path, 'inputs')
  File "/home/pi/Mycodo/mycodo/utils/modules.py", line 16, in load_module_from_file
    module_custom = importlib.util.module_from_spec(spec)
  File "<frozen importlib._bootstrap>", line 580, in module_from_spec
AttributeError: 'NoneType' object has no attribute 'loader'

ReplyQuote
Kyle Gabriel
(@kylegabriel)
Member Admin
Joined: 6 years ago
Posts: 570
 
Posted by: @stevecos

I added an extra line into the on_message function, line 215, of /home/pi/Mycodo/mycodo/inputs/mqtt_paho.py

Rather than editing the built-in module, you should copy it to a new file on your local filesystem, change the unique ID (as well as the name to distinguish it from the original), then import it on the Configure -> Custom Input page. Built-in inputs are overwritten during an upgrade, where imported Inputs are not.

Posted by: @stevecos

I went to Data->Data to add another data point, but that through an exception. 

Never had that happen. Could be related to your modification(s) to the internal files?

Posted by: @stevecos

"analog_sensor,null=5289.0" is never going to be a float! 

I'm not too familiar with Cayenne. Are you not able to define the output to return just a numerical value if you query your analog_sensor?

Mycodo Developer


ReplyQuote
Kyle Gabriel
(@kylegabriel)
Member Admin
Joined: 6 years ago
Posts: 570
 
Posted by: @stevecos

Could you please point me to a page where I can create some suggested documentation?  (Which will give me an idea of an appropriate format.)

https://kizniche.github.io/Mycodo/Supported-Inputs/#mycodo-mqtt-subscribe-paho is where you will find the documentation specific for the MQTT Input. If you look around the supported Input page, you will see some have descriptions. This entire page is auto-generated from information contained in the Inputs, with the description being taken from the 'message' key of the INPUT_INFORMATION dictionary. Now, It's probably not advisable to include info specific to Cayenne in this area, but you can create a new section in the documentation on the Inputs section, similar to how The Things Network is discussed ( https://kizniche.github.io/Mycodo/Inputs/#the-things-network ). The doc is located at https://github.com/kizniche/Mycodo/blob/master/docs/Inputs.md

Mycodo Developer


ReplyQuote
Kyle Gabriel
(@kylegabriel)
Member Admin
Joined: 6 years ago
Posts: 570
 

If for some reason it's not possible to modify the return value, it might be preferable to rename the original MQTT input to something like "MQTT (value)" and duplicate and modify it to create a second like "MQTT (Cayenne)" or similar, which can have delimiters, filters, etc. set by the user in order to extract the relevant value from the return string.

Mycodo Developer


ReplyQuote
andris_mz
(@andris_mz)
Active Member
Joined: 1 month ago
Posts: 8
 

Greetings to all!
I wrote a week ago about the problem of reading MQTT data, but so far I have not received any response.
Although there are many discussions in the forum.

Have any of the forum readers really encountered the following: MQTT reads data from esp32 or Node-red, everything works.
As soon as MQTT_output on/off or MQTT_output value is activated, its data reading is stopped and does not resume.
At the same time, the MQTT_output function works.
My previous topic in the forum as I configured Mycodo: https://kylegabriel.com/forum/general-discussion/mqtt-data-reading-problem-from-esp32/#post-1074

Please provide an answer and advice on this problem.
From the log file, if I understand correctly that the MQTT input is switched off after activating the MQTT output.

Sincerely,
Andris

 

2021-01-31 19:57:54,071 - INFO - mycodo.daemon - Mycodo daemon v8.8.8 starting
2021-01-31 19:57:54,770 - INFO - mycodo.outputs.mqtt_3b6f5df5 - Initialized in 171.6 ms
2021-01-31 19:57:54,879 - INFO - mycodo.outputs.mqtt_value_24a81a27 - Initialized in 107.4 ms
2021-01-31 19:57:54,880 - INFO - mycodo.controllers.controller_output - Activated in 555.3 ms
2021-01-31 19:57:55,381 - INFO - mycodo.daemon - All activated Conditional controllers started
2021-01-31 19:57:55,381 - INFO - mycodo.daemon - All activated Trigger controllers started
2021-01-31 19:57:55,737 - DEBUG - mycodo.inputs.mqtt_paho_751d0e5a - inp_117r Client created with ID mycodo_mqtt_client
2021-01-31 19:57:55,737 - DEBUG - mycodo.inputs.mqtt_paho_751d0e5a - inp_r128 Connecting MQTT callback functions
2021-01-31 19:57:55,738 - DEBUG - mycodo.inputs.mqtt_paho_751d0e5a - inp_r133 MQTT callback functions connected
2021-01-31 19:57:55,738 - INFO - mycodo.inputs.mqtt_paho_751d0e5a - inp_r144 Connected to 192.168.1.200 as mycodo_mqtt_client
2021-01-31 19:57:55,739 - DEBUG - mycodo.inputs.mqtt_paho_751d0e5a - r155 Subscribed to MQTT channel 'paho/node/integer'
2021-01-31 19:57:55,740 - DEBUG - mycodo.inputs.mqtt_paho_751d0e5a - r155 Subscribed to MQTT channel 'esp32/temperature'
2021-01-31 19:57:55,740 - DEBUG - mycodo.inputs.mqtt_paho_751d0e5a - r155 Subscribed to MQTT channel 'esp32/humidity'
2021-01-31 19:57:55,742 - DEBUG - mycodo.inputs.mqtt_paho_751d0e5a - r162 Connected to 'None', rc: 0
2021-01-31 19:57:55,743 - DEBUG - mycodo.inputs.mqtt_paho_751d0e5a - inp_r166 Subscribing to mqtt topic: None, 1, (0,)
2021-01-31 19:57:55,743 - DEBUG - mycodo.inputs.mqtt_paho_751d0e5a - inp_r166 Subscribing to mqtt topic: None, 2, (0,)
2021-01-31 19:57:55,743 - DEBUG - mycodo.inputs.mqtt_paho_751d0e5a - inp_r166 Subscribing to mqtt topic: None, 3, (0,)
2021-01-31 19:57:55,744 - INFO - mycodo.controllers.controller_input_751d0e5a - Activated in 201.7 ms
2021-01-31 19:57:55,744 - INFO - mycodo.daemon - All activated Input controllers started
2021-01-31 19:57:55,745 - INFO - mycodo.daemon - All activated Math controllers started
2021-01-31 19:57:55,746 - INFO - mycodo.daemon - All activated PID controllers started
2021-01-31 19:57:55,746 - INFO - mycodo.daemon - All activated LCD controllers started
2021-01-31 19:57:55,746 - INFO - mycodo.daemon - All activated Custom controllers started
2021-01-31 19:57:55,995 - DEBUG - mycodo.inputs.mqtt_paho_751d0e5a - inp_r174 Message received: Channel: paho/node/integer, Value: 37
2021-01-31 19:57:55,999 - DEBUG - mycodo.inputs.mqtt_paho_751d0e5a - inp_r222 Adding measurement to influxdb: {0: {'measurement': 'volume', 'unit': 'l', 'value': 37.0, 'timestamp_utc': datetime.datetime(2021, 1, 31, 17, 57, 55, 994942)}}
2021-01-31 19:57:56,017 - DEBUG - mycodo.inputs.mqtt_paho_751d0e5a - inp_r174 Message received: Channel: esp32/temperature, Value: 24.51
2021-01-31 19:57:56,020 - DEBUG - mycodo.inputs.mqtt_paho_751d0e5a - inp_r222 Adding measurement to influxdb: {1: {'measurement': 'temperature', 'unit': 'C', 'value': 24.51, 'timestamp_utc': datetime.datetime(2021, 1, 31, 17, 57, 56, 17081)}}
2021-01-31 19:57:56,075 - DEBUG - mycodo.inputs.mqtt_paho_751d0e5a - inp_r174 Message received: Channel: esp32/humidity, Value: 26.40
2021-01-31 19:57:56,078 - DEBUG - mycodo.inputs.mqtt_paho_751d0e5a - inp_r222 Adding measurement to influxdb: {2: {'measurement': 'humidity', 'unit': 'percent', 'value': 26.4, 'timestamp_utc': datetime.datetime(2021, 1, 31, 17, 57, 56, 75173)}}
2021-01-31 19:57:56,139 - INFO - mycodo.controllers.controller_widget - Activated in 392.3 ms
2021-01-31 19:57:56,988 - DEBUG - mycodo.inputs.mqtt_paho_751d0e5a - inp_r174 Message received: Channel: paho/node/integer, Value: 38
2021-01-31 19:57:56,991 - DEBUG - mycodo.inputs.mqtt_paho_751d0e5a - inp_r222 Adding measurement to influxdb: {0: {'measurement': 'volume', 'unit': 'l', 'value': 38.0, 'timestamp_utc': datetime.datetime(2021, 1, 31, 17, 57, 56, 988683)}}
2021-01-31 19:57:57,016 - DEBUG - mycodo.inputs.mqtt_paho_751d0e5a - inp_r174 Message received: Channel: esp32/temperature, Value: 24.51
2021-01-31 19:57:57,019 - DEBUG - mycodo.inputs.mqtt_paho_751d0e5a - inp_r222 Adding measurement to influxdb: {1: {'measurement': 'temperature', 'unit': 'C', 'value': 24.51, 'timestamp_utc': datetime.datetime(2021, 1, 31, 17, 57, 57, 16193)}}
2021-01-31 19:57:57,027 - DEBUG - mycodo.inputs.mqtt_paho_751d0e5a - inp_r174 Message received: Channel: esp32/humidity, Value: 26.44
2021-01-31 19:57:57,031 - DEBUG - mycodo.inputs.mqtt_paho_751d0e5a - inp_r222 Adding measurement to influxdb: {2: {'measurement': 'humidity', 'unit': 'percent', 'value': 26.44, 'timestamp_utc': datetime.datetime(2021, 1, 31, 17, 57, 57, 27832)}}
2021-01-31 19:57:57,141 - INFO - mycodo.daemon - Mycodo daemon started in 3.069 seconds
2021-01-31 19:57:57,142 - INFO - mycodo.daemon - 43.63 MB RAM in use
2021-01-31 19:57:57,989 - DEBUG - mycodo.inputs.mqtt_paho_751d0e5a - inp_r174 Message received: Channel: paho/node/integer, Value: 39
2021-01-31 19:57:57,992 - DEBUG - mycodo.inputs.mqtt_paho_751d0e5a - inp_r222 Adding measurement to influxdb: {0: {'measurement': 'volume', 'unit': 'l', 'value': 39.0, 'timestamp_utc': datetime.datetime(2021, 1, 31, 17, 57, 57, 989550)}}
2021-01-31 19:57:58,005 - DEBUG - mycodo.inputs.mqtt_paho_751d0e5a - inp_r174 Message received: Channel: esp32/temperature, Value: 24.51
 ****
2021-01-31 19:58:08,999 - DEBUG - mycodo.inputs.mqtt_paho_751d0e5a - inp_r222 Adding measurement to influxdb: {0: {'measurement': 'volume', 'unit': 'l', 'value': 2.0, 'timestamp_utc': datetime.datetime(2021, 1, 31, 17, 58, 8, 996874)}} 2021-01-31 19:58:09,017 - DEBUG - mycodo.inputs.mqtt_paho_751d0e5a - inp_r174 Message received: Channel: esp32/temperature, Value: 24.48 2021-01-31 19:58:09,020 - DEBUG - mycodo.inputs.mqtt_paho_751d0e5a - inp_r222 Adding measurement to influxdb: {1: {'measurement': 'temperature', 'unit': 'C', 'value': 24.48, 'timestamp_utc': datetime.datetime(2021, 1, 31, 17, 58, 9, 17463)}} 2021-01-31 19:58:09,025 - DEBUG - mycodo.inputs.mqtt_paho_751d0e5a - inp_r174 Message received: Channel: esp32/humidity, Value: 26.59 2021-01-31 19:58:09,029 - DEBUG - mycodo.inputs.mqtt_paho_751d0e5a - inp_r222 Adding measurement to influxdb: {2: {'measurement': 'humidity', 'unit': 'percent', 'value': 26.59, 'timestamp_utc': datetime.datetime(2021, 1, 31, 17, 58, 9, 24994)}} 2021-01-31 19:58:09,245 - DEBUG - mycodo.outputs.mqtt_3b6f5df5 - output_on_off(on, 0, sec, 3.0, 0.0, True) 2021-01-31 19:58:09,247 - DEBUG - mycodo.inputs.mqtt_paho_751d0e5a - inp_r229 input Disconnected result code 1 2021-01-31 19:58:09,249 - DEBUG - mycodo.outputs.mqtt_3b6f5df5 - Output 3b6f5df5-19c9-4bde-a312-34e2370ff978 CH0 (esp32/output) on for 3.0 seconds. Output returned: None 2021-01-31 19:58:10,249 - DEBUG - mycodo.inputs.mqtt_paho_751d0e5a - r162 Connected to 'None', rc: 0 2021-01-31 19:58:12,250 - DEBUG - mycodo.outputs.mqtt_3b6f5df5 - output_on_off(off, 0, None, 0.0, 0.0, True) 2021-01-31 19:58:12,254 - DEBUG - mycodo.inputs.mqtt_paho_751d0e5a - inp_r229 input Disconnected result code 1 2021-01-31 19:58:12,256 - DEBUG - mycodo.outputs.mqtt_3b6f5df5 - Output 3b6f5df5-19c9-4bde-a312-34e2370ff978 CH0 (esp32/output) OFF at 2021-01-31 19:58:12. Output returned: None 2021-01-31 19:58:13,255 - DEBUG - mycodo.inputs.mqtt_paho_751d0e5a - r162 Connected to 'None', rc: 0

ReplyQuote
Kyle Gabriel
(@kylegabriel)
Member Admin
Joined: 6 years ago
Posts: 570
 

Please keep discussions separate to their own respective threads. When you make a new reply, it will bring the topic back to the top of the forum. I receive a lot of messages and I can't always respond to them all in a timely manner, especially if I don't have the time or means to test, investigate, or learn and understand them. I understand your frustration, but including information in this thread about another issue only adds to confusion and delay in finding a solution, because now users who might be able to contribute given the new data you presented will not find it in the original thread documenting the issue.

Mycodo Developer


ReplyQuote
andris_mz
(@andris_mz)
Active Member
Joined: 1 month ago
Posts: 8
 

Thanks Kyle for giving an answer. 🤗 
I certainly apologize for writing my problem in this discussion. But here, too, we are talking about MQTT.
I wrote my question and additional data (Flow for Node-red) again here: https://kylegabriel.com/forum/general-discussion/mqtt-data-reading-problem-from-esp32/#post-1202

I look forward to hearing from You!
Sincerely, Andris


ReplyQuote
stevecos
(@stevecos)
Active Member
Joined: 1 month ago
Posts: 13
Topic starter  

Cayenne data is coming very happily now.  Most important to me is the simple, reliable, visualization combined with being able to store the data indefinitely and the researcher this project is for can download the data easily.  Thank you again for putting this together.
My sensor values can be viewed at Mycodo - Default Dashboard - Mycodo 8.8.8 (iotdata.nz)  Guest/PasswordPW (nothing sensitive about it as far as I know).

Outstanding issues:

  • The one we have already discussed - we need a specific modification to cope with the way Cayenne sends MQTT data.
  • Cayenne uses a unique string followed by a channel number, with embedded text, to identify each MQTT channel. The way Mycodo data sources are defined, this very long indication is also used to name each data point in the graph - looking at the dashboard above, you can see that the values are hard to read.
  • There is also the issue of units - Battery is in millivolts, volts would be better, and each temperature needs to be divided by 100 to get degrees Celsius.

Can Math data units help me with this?  I've created an experimental graph below the main one.  Will play with it tomorrow.

😀


ReplyQuote
Kyle Gabriel
(@kylegabriel)
Member Admin
Joined: 6 years ago
Posts: 570
 

I'm unable to log in with the credentials you provided, so I can't see the issue with the name. As for conversions, all input channels should be able to be converted to alternate units, so long as a conversion exists or you've added the desired conversion. 

Mycodo Developer


ReplyQuote
stevecos
(@stevecos)
Active Member
Joined: 1 month ago
Posts: 13
Topic starter  

Thanks!
Have fixed the password and attached a photo too.

:-)


ReplyQuote
Page 1 / 2