MQTT | What is MQTT | MQTT in Depth | QoS | FAQs | MQTT Introduction

Hi visitor please read full article on my blog https://iotbyhvm.ooo/mqtt/

What is MQTT ?

MQTT used to stand for MQ Telemetry Transport, but is today referred to simply as MQTT and is no longer an acronym. It is an extremely simple and lightweight Publish/Subscribe messaging protocol invented at IBM and Arcom (now Eurotech) to connect restricted devices in low bandwidth, high-latency or unreliable networks.

Recommended: MQTT Servers/Brokers | MQTT Public Brokers List

Constrainted devices in unreliable networks

MQTT was originally designed to connect sensor nodes over communication networks that are unreliable or high-latency, or both.

As an example, MQTT has long been and still is used for monitoring oil and gas pipeline operations. Pipelines don’t always run through areas with good connectivity, requiring a communication protocol that can cope with such an environment. Seensor nodes are often quite constrained in terms of resources and abilities, running on battery power and forced to limit energy consumption. They might also rely on satellite communication links with limited bandwidth and performance. MQTT was designed for such environments.

It’s pretty clear today that MQTT is not limited to its original oil and gas use case, but has much broader applications in M2M, mobile and IoT verticals. Facebook uses MQTT in their Facebook Messenger. That seems very different to monitoring a pipeline first, but there are many similar technical constraints, like a mobile device with limited power and rapid network connectivity changes.

To choose the right messaging technology, it is critical to know in what kind of environment the devices will be running and what their communication patterns will look like.

If your project shares some similarities with the uses cases just mentioned, with a Publish/Subscribe messaging pattern, or just the need for a Message Queue, read on! There are good chances MQTT is the right tool for the job.

---

MQTT in Depth

---

Quality of Service

Quality of Service (QoS) is a rather broad topic in general. For MQTT though, QoS takes a simple form. It allows an application (= a client publishing or subscribing to a topic) to choose one of the following transport guarantees:

Fire and forget (QoS 0)

A client won’t receive any confirmation from the broker upon receipt. Likewise a message delivered to a client from the broker is not required to be acknowledged. This is the fastest way to publish and receive messages, but also the one where message loss is most likely to happen.

At least once (QoS 1)

A client will receive a confirmation message from the broker upon receipt. If the expected confirmation is not received within a certain time frame, the client has to retry the message. A message received by a client must be acknowledged on time as well, otherwise the broker will re-deliver the message.

Exactly once (QoS 2)

The first part of the QoS 2 flow is similar to QoS 1. A second flow ensures that sender and receiver (broker) agree on where the message stands. The point is to avoid duplicate routing, i.e. delivery to the subscribers downstream of the broker, as the broker can recognize duplicates itself. QoS 2 doesn’t necessarily imply that the publisher will only send the message once, it will still retry if it gets no acks from the broker.

The choice of QoS should be based on the network environment. A mobile app could use QoS 0 whenever it is connected to a reliable wireless connection and falling back to QoS 1 when connected to the mobile network.

Recommended: IoT Protocols | MQTT Products (that use MQTT)

Last Will Testament

This feature is unique among IoT messaging protocols. The Last Will Testament or short LWT lets a client provide a testament along with its credentials when connecting to the broker. If the client disconnects ungracefully at some point later (maybe because his power source died), it can let the broker deliver a message to other clients. (Comparable to a real testament that gets communicated to the bereaved). This LWT message has the same form as an ordinary message and gets routed via the same mechanics.

Such a simple LWT mechanism can become very handy if your application has to track some sort of presence status of the participating devices.

Clean Session

To understand what the clean session feature is about, we have to look at the MQTT session initiation. When a device connects to a MQTT broker for the first time it implicitly creates a new MQTT session. One part of that session consists of state stored on the broker, and the other part of the session is state stored on the client.

Session state can be bound to the lifetime of the TCP connection, but it doesn’t have to be. The state on the broker and the client can also be persisted.

What is in the Session State? Let’s have a look what the MQTT specification says about what should be in the server state and the client state of a session:

The session state in the client consists of:

• QoS 1 and QoS 2 messages which have been sent to the Server, but have not been completely acknowledged.
• QoS 2 messages which have been received from the Server, but have not been completely acknowledged.

The session state in the server consists of:

• The existence of a Session, even if the rest of the Session state is empty.
• The Client’s subscriptions.
• QoS 1 and QoS 2 messages which have been sent to the Client, but have not been completely acknowledged.
• QoS 1 & QoS 2 messages pending transmission to the Client.
• QoS2 messages which have been received from the Client, but have not been completely acknowledged.
• Optionally, QoS 0 messages pending transmission to the Client.

In conclusion, one of the benefits of MQTT and its session concept, is that sessions can survive re-connects. When connectivity is very expensive, limited, or rapidly changing a device will not be connected all the time. While offline, the broker still keeps the session around and accumulates all messages that match the device’s subscriptions. As soon as the client reconnects all the outstanding messages will be delivered.

Retained Messages

Publishers can mark any message they send as ‘to be retained’. There can only be one retained message per topic, and any new ‘retained’ message by any publisher to that topic will replace the current one. Retained messages are only delivered when a client subscribes to a topic. The most common use case for this is to push some kind of last known message (or sensor value) to new subscribers. The broker has to store all retained messages in a database, as the retained message will not expire.

Retained messages are not to be confused with ‘offline’ messages, i.e. the QoS 1 and 2 messages that the broker has to store for any offline subscribers.

This feature paired with the LWT feature enable to implement a simple presence notification system using plain MQTT.

Recommended: STOMP — Simple/Streaming Text Oriented Messaging Protocol

Retained Messages

Publishers can mark any message they send as ‘to be retained’. There can only be one retained message per topic, and any new ‘retained’ message by any publisher to that topic will replace the current one. Retained messages are only delivered when a client subscribes to a topic. The most common use case for this is to push some kind of last known message (or sensor value) to new subscribers. The broker has to store all retained messages in a database, as the retained message will not expire.

Retained messages are not to be confused with ‘offline’ messages, i.e. the QoS 1 and 2 messages that the broker has to store for any offline subscribers.

This feature paired with the LWT feature enable to implement a simple presence notification system using plain MQTT.

Message Queue

A Message Queueing system receives, stores and forwards messages. Those actions tend to vary heavily from one implementation or messaging protocol to another.

The core component of a queueing system is, of course, a data structure called queue. The underlying theory of a queueing system is quite complex, so let’s assume for now that a queue is some sort of a database. While most general purpose databases have powerful query capabilities such as the SQL query language, a queue in it’s most basic form only knows two operations, push and pop.

The Push operation inserts a new element to the queue, the Pop operation reads the oldest element from the queue and deletes it afterwards. This is also called the FIFO model, first in first out. In certain cases the Pop operation should always read the newest element instead of the oldest. Such a datastructure is called a stack, while the model is called LIFO, last in first out.

A message queueing system could implement more advanced queuing operations like message priority support or ways to automatically delete old messages.

A message queue is a perfect addition to the Publish/Subscribe pattern because it backs the decoupling principle. For this reason most Publish/Subscribe protocol implementations rely on a message broker that also implements a message queuing system.

Publish/Subscribe

Publish/Subscribe is a messaging pattern that aims to decouple the sending (Publisher) and receiving (Subscriber) party. A real world example could be a sport mobile app that shows you up-to-date information of a particular football game you’re interested in. In this case you are the subscriber, as you express interest in this specific game. On the other side sits the publisher, which is an online reporter that feeds a system with the actual match data. This system, which is often referred as the message broker brings the two parties together by sending the new data to all interested subscribers.

The Publish/Subscribe messaging pattern brings many benefits:

• Implement the publisher and subscriber parties independently from each other
• Publishers and Subscribers don’t require to know each other
• One Subscriber could receive from many different Publishers
• One Publisher could send data to many different Subscribers

The benefits can be sumarized to Loose Coupling, the decoupling of publisher and subscriber, as well as Scalability through the parallel nature of the publisher and subscriber components.

Many standardized messaging protocols that implement a Publish/Subscribe pattern exist. In the area of application level protocols the most interesting ones are:

• AMQP, Advanced Message Queueing Protocol
• MQTT, MQ Telemetry Transport
• JMS, Java Messaging Service
• XMPP, Extensible Messaging and Presence Protocol

A publish/subscribe messaging protocol is only as useful as its message broker implementations.

FAQs

What is MQTT?

MQTT is a lightweight publish/subscribe messaging protocol designed for M2M (machine to machine) telemetry in low bandwidth environments. It works on top of the TCP/IP protocol and it needs a central MQTT Broker to collect and send data from and to publishing and subscribing machines.

What is a MQTT broker?

MQTT Broker is a central server or middle-ware which act as an interface between publisher & subscriber and used to collect the data from the publisher device and send it to the subscribers device. A device wants to send data to the broker, it is called a publisher and a device wants to receive data from the broker, it is called as subscriber

Recommended: How To Create Secure MQTT Broker

Why MQTT is used in IoT?

MQTT is a lightweight, machine to machine communications protocol which helps the small IoT device in low bandwidth or network to send or publish data to the MQTT Broker or server. MQTT protocol provides faster response and throughput, lower battery and bandwidth usage, and work in low or poor network connectivity. It also provides secure communication via TLS/SSL, edge level authentication and QoS to manage data traffic to reduce packet loss and latency on the network. This advantages make MQTT well-suited for “internet of things” applications

Does MQTT require Internet?

Internet is not needed when you connect the MQTT broker and your client or device under the same network. But if you want to connect a device from a different network, you must host the MQTT Broker in cloud and connect your device via internet using Public Ip address. We recommend hosting the Broker securely on the cloud for controlling devices from anywhere.

Is MQTT secure?

Yes, MQTT Broker is built with TLS/SSL and Edge device authentication. TLS/SSL which provides a secure mechanism and data encryption between device & MQTT broker and edge device authentication will help the edge device to connect with username and password for secure data transfer.

Recommended: Setting up Authentication in Mosquitto MQTT Broker

Does MQTT Broker use TCP or UDP?

MQTT Broker use TCP/IP protocol to establish connection to client or device. However, for devices which are connected under UDP or device which are using zigbee or bluetooth protocol can be connected to the Broker via MQTT-SN Gateway.

What port does MQTT use?

MQTT has two types of communication methodology. TCP and Websocket. For TCP Internet Assigned Numbers Authority (IANA) reserved TCP/IP port ‘1883’ for a unencrypted and TCP/IP port ‘8883’ for TLS/SSL communication. You can use any flexible port for the MQTT over WebSocket.

Is MQTT Broker store data?

The specification does not mandate it. But the brokers need to store data as the manager application needs to use the data. Bevywise MQTT Broker store mqtt data in database like SQLite[default], MySQL, Elastic search and Mongo DB. Users can also store the data to any database using Custom implementation.

How can I view the connected device and its data in MQTT Broker?

Users can view the all connected device and its published data in MQTT Broker user interface and dashboard. MQTT Broker also has an individual web view for each active and inactive devices. Users can view the status of each devices, Broker uptime, Error log, add authentication credentials [username and password], active and inactive topics and also download Client librairies from user interface. MQTT Broker user interface runs in 8080 port and users can access via “http://< server ip address or localhost >:8080”

Does MQTTRoute has Rule based automation?

Yes, MQTTRoute has a built-in rule engine which user an create rules or alert based on published messages. Rule engine help the users to send alert messages or forward published message to the subscriber when the data from a sensor crosses a certain threshold or range.

How can I build my IOT application over MQTT Broker ?

It is not advisable to build your IOT application via MQTT. You have to integrate using REST API. With this user can integrate their web application and get details about connected devices, authentication details etc by sending API request to MQTT Broker and the MQTTBroker send back data based on the API request or call.

Can I integrate MQTT Broker to my data visualization tool?

Yes, you can integrate MQTT Broker to your data visualization tool via Custom Implementation. Through custom implementation you can write your algorithm or code and push the data to your data visualization tool.

Is MQTT Broker can be hosted in containers?

Yes, MQTT Broker can be hosted in Docker containers.

You may like also:

• How To Create Secure MQTT Broker
• DHT11 sensor with NodeMCU using Arduino IDE
• Simple Raspberry Pi Home Security System
• Using Mq135 Sensor with InfluxDB
• Best OS for Raspberry Pi
• ROCK Pi 4 : Overview | Installation
• Remote control your Raspberry Pi from your PC with VNC!
• How To Use Raspberry pi in a truely headless mode