Update module github.com/wind-c/comqtt/v2 to v2.6.7 #95
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "renovate/github.com-wind-c-comqtt-v2-2.x"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
This MR contains the following updates:
v2.6.5→v2.6.7Release Notes
wind-c/comqtt (github.com/wind-c/comqtt/v2)
v2.6.7Compare Source
What's Changed
Details
#183 fix: grow packet buffers as data arrives instead of pre-allocating
A packet body's size comes from a
Remaining Lengththe peer declares in the fixed header, before a single body byte has arrived.ReadPacketallocated that much up front and then blocked on the read. An unauthenticated peer could declare the protocol maximum of 256 MB, send five bytes, and go quiet, and the broker would hold that much memory for as long as the socket stayed open.The
MaximumPacketSizecheck could not stop it: it is keyed off the same declared value, the shipped default is0("unlimited"), andReadFixedHeaderskips the check entirely when it is0. No read deadline is in effect while the CONNECT packet itself is being read either, so nothing else would have timed the connection out.Bodies over 64 KiB are now grown geometrically as data actually arrives, so broker memory tracks bytes received rather than bytes declared. Bodies up to 64 KiB keep the previous single-allocation path, byte for byte.
Measured on this tree (Windows, Go 1.26, unauthenticated MQTT 5, one connection per 256 MB declaration, process working set):
Flat, and still flat when the connections are held open. A peer that declares a large body and then sends nothing still reserves 64 KiB while it waits, which is the same order as the goroutine and socket the connection already costs.
The read error semantics now match
io.ReadFullexactly, including the case where a read delivers the final bytes together withio.EOF. That case is reachable in production:crypto/tlsreturns(n, io.EOF)from a singleReadwhen a close_notify arrives behind the last data.The same change also fixes a leaked registration: a client that could not receive its CONNACK stayed in the client registry forever, holding its buffer and its connection.
+362/−10 across
mqtt/clients.go,mqtt/server.goand the tests, including an equivalence test that pins the new read path to the behaviour of theio.ReadFullit replaced, and a test that proves the growth path consumes exactly the declared body and no more.#181 fix: a partial capabilities block no longer zeroes every default
The presence of any
capabilities:key in the config made YAML allocate a zero-valued struct, soCapabilitiesstopped being nil andNew()skippedDefaultServerCapabilitiesentirely. Declaring a single key then silently zeroed every other capability.The worst of them was
MaximumMessageExpiryInterval: at zero, the once-per-second expiry sweep matched every queued message as expired, so the entire offline queue was discarded and persistent sessions stopped delivering, with nothing logged.Capabilities.UnmarshalYAMLnow decodes on top of the defaults, merging per key rather than per field, so a partial block keeps the defaults for the keys it omits while explicit zeros (maximum-packet-size: 0, meaning unlimited) still work.This is a behaviour change for anyone who relied on a partial block zeroing the other capabilities. If you wanted a capability off, set it explicitly.
+188/−9 across
mqtt/server.goand the tests.#180 fix: bound repeated MQTT v5 property counts
MQTT v5 puts no count limit on repeated User Property or Subscription Identifier entries, and neither has a count limit in the specification. Left unbounded, a single packet amplifies badly:
Properties.Useris a slice of 32-byte entries, so a 10 MB property section carrying two-byte properties grew to about 46 MB of slice, andProperties.Copyduplicated all of it again on top of the packet buffers. One unauthenticated CONNECT could push a broker's RSS up by two orders of magnitude.Both are now capped: 128 User Properties and 1024 Subscription Identifiers per packet. The subscription identifier limit is deliberately much higher, because MQTT v5 permits one identifier per filter in a SUBSCRIBE and a conforming client needs as many as it has filters.
+150/−2 across
mqtt/packets/, with tests for both limits.#182 fix: cap topic filter depth to stop trie memory amplification
A retained message allocated one trie node per
/separated topic level, so a deeply nested topic name could be amplified into a large structure from a single message. The depth is now capped, and a filter that exceeds it is rejected rather than truncated, because a truncated filter no longer matches what the client asked for. [MQTT-4.7.3-1]+342 across
mqtt/server.go,mqtt/topics.goand the tests.#186 fix: retry transient Accept errors instead of silently killing a listener
Listeners.Servecalls a listener'sServeexactly once, with nothing to restart it, so returning on a transient accept failure took the listener down for good, and nothing was logged. The OS listening socket stays open, so clients kept completing the TCP handshake and then hung until their own timeout, each occupying an accept queue slot.Accept now retries with a capped backoff. The most common causes were EMFILE when a burst of connections exhausted the file descriptor table, and ECONNABORTED when a peer disappeared mid-handshake. The retry is bounded and the listener is only torn down when it is actually closed.
+461/−33 across
mqtt/listeners/, with a test that a client is served normally after a transient accept error.#191 fix: drop a relayed publish whose declared body is longer than its payload
A cluster peer chooses the bytes in a relayed publish, including the remaining length it declares. An empty payload made
readFixedHeaderreadb[0], and a payload shorter than the body it declared made the offset negative and the slice out of range. Both panicked inside the relay worker.The relay now drops a payload it cannot unpack. A malformed relay was already dropped silently when the fixed header could not be decoded, so this keeps the same behaviour and does not add a log line that a hostile peer could use to flood the log.
Note that the gRPC channel between nodes is unauthenticated and unencrypted. That is a separate and larger question, and this change only makes the relay resilient to malformed input.
+105 across
cluster/agent.goand the tests.#184 fix: break inflight resend order ties so persistent sessions resend in order
Createdis a unixtime with one-second resolution, so messages published within the same second tie.sort.Sliceis unstable and the slice came out of a map with randomised iteration order, so tied messages were left in an arbitrary order that changed on every call, and resends of a persistent session could go out rotated rather than in their original order. [MQTT-4.6.0-1]Packet ids are handed out in ascending order, are unique, and are preserved across resend and session restore, so they make this a strict total order.
+181/−1 across
mqtt/inflight.goand the tests.#185 fix: make same-id session takeover atomic
Two connections with the same client id could each believe it had won the takeover. The registry now removes the displaced client with a compare-and-delete, so a stale cleanup cannot drop a newer connection with the same id.
+154/−2 across
mqtt/clients.go,mqtt/server.goand the tests.#187 docs: explain accept queue limits and reconnect bursts
A listening TCP socket holds only a limited number of connections that completed the handshake but were not handed to the process. The depth is a system setting, and comqtt does not override it. The README now covers it, with the
somaxconnsettings for Linux and macOS, and why clients should reconnect with exponential backoff plus jitter.+35 in
README.md.#189 and #190 test: make the publish tests deterministic
Twelve tests closed the client connection a fixed time after publishing and then asserted on the bytes that arrived, so they passed or failed depending on how quickly the write loop drained the outbound queue. They now wait for the writes to complete, which is a delivery signal rather than a timing guess: a write to a pipe completes only once the reader has taken the bytes.
Three of them had assertions too weak to notice a broken write, and one of them had lost a length check that was implicit in the original. A test that compared the message expiry interval also failed about once in 20000 runs, because the interval is derived from the time the publish was created and the clock can tick between the two readings.
The whole
mqttpackage now passes 50 consecutive runs under-racewith no failures.Upgrade notes
No configuration change is required.
maximum-packet-sizeis still0by default, which means unlimited, and the packet buffer no longer follows the declared length.Two behaviour changes are worth knowing about:
capabilities:block now merges on top of the defaults instead of zeroing them. If you wanted a capability disabled, set it explicitly.Thank you
Thanks to everyone who reported an issue or reviewed a change in this cycle.
v2.6.6Compare Source
What's Changed
extraEnvpassthrough forCOMQTT_*Details
#174 fix: Inflight resend deadlock, quota overshoot and resend ordering
Three concurrency/correctness defects in
mqtt/inflight.go, all found while load-testing many QoS 1 publishers against one slow subscriber.NextImmediate()tookRLockand then calledGetAll(), which tookRLockagain. Go'sRWMutexblocks new readers as soon as a writer is queued, so a publisher'sSetlanding between the two read locks left the goroutine holding the first lock forever.processPackethits that path on every packet when inflight is non-empty (server.go:714), so a single client could wedge permanently — no PUBACK processing, unbounded inflight growth, goroutine leak. Reproduced reliably before the fix; passes in ~0.2s after.if Load() > 0 { Add(-1) }), which is not atomic. Concurrent publishers could push the send quota below zero or pastmaximumSendQuota, transiently over-running the client'sReceiveMaximum. Now CAS loops, so0 <= quota <= maxactually holds.GetAllsorted byuint16(Created), butCreatedis an int64 unix timestamp. The truncation wraps every ~18 hours and scrambled resend order across the boundary. Now compares the int64s directly.+138/−11 across
mqtt/inflight.goandmqtt/inflight_test.go, with regression tests for all three.#172 feat: environment variable overrides on top of YAML
config.Loadnow appliesCOMQTT_*environment variables on top of the YAML baseline, so operators can override scalar values at deploy time without rewriting the config file — e.g. injecting a dynamic Kubernetes pod IP intocluster.bind-addrwith Hashicorp Raft. Refs #119.COMQTT_+ the dotted yaml path, uppercased,-→_(e.g.mqtt.tcp→COMQTT_MQTT_TCP)string,int/uintandboolcluster.members) are not overridable and keep their YAML value+328/−3, with 13 test cases covering baseline-unchanged, override, supply-a-missing-value, empty-string-clearing, unset-leaves-untouched, and the #119 bind-addr scenario.
#173 fix: routing table lost on raft snapshot restore
Since
d5c33e1,KV.GetAll()returns a copy — so both restore paths (hashicorpFsm.Restoreand etcdrecoverFromSnapshot) were decoding snapshots into a throwaway map. A node that installed a snapshot ended up with an empty routing table: remote subscriptions stopped receiving forwarded publishes until they re-subscribed. Silent failure, high diagnostic cost.Adds
KV.Restore, which decodes into a fresh map and swaps it in under the lock, replacing prior state as raft snapshot semantics require. Covered by new tests for both backends, including a corrupt snapshot.#175 chore(helm):
extraEnvpassthrough, chart 0.4.0 → 0.5.0#172's env overrides were unreachable from Helm: the chart only injected its own seven hardcoded variables and exposed no way to pass anything else. Adds an
extraEnvlist rendered into the broker container in bothsingleandclustermodes, appended after the chart's downward-API variables. Defaults to[], so existing releases render unchanged.Also bumps
appVersion2.6.0→2.6.6. Chart version is independent of the broker version; this is the fourth chart release.#169 #170 security: ACL bypass via prefix topic matching (CVSS 4.3)
Two separate authorisation defects in
MatchTopic(mqtt/hooks/auth/ledger.go). Reported by Team Atlanta.MatchTopicreturnedtrueafter exhausting filter parts without checking whether topic parts remained, so a literal ACL filter such asallowedalso matched publishes toallowed/secret. Matching now requireslen(topicParts) == len(filterParts).#wildcard non-compliance. The bounds check returnedfalsebefore testing whether the filter part was#, so the multi-level wildcard could not match zero sublevels as MQTT 4.7 requires. NowMatchTopic('a/#', 'a')istruewhileMatchTopic('a/+', 'a')staysfalse.#171 chore: raft log typos
Eight message corrections in the etcd backend (
dnapshot→snapshot,failed ti listen→failed to listen,listeing→listening,membe→member,snapshit→snapshot). Contributed by @MattFiocca.⚠️ Action required: ACL rule behaviour change
The
#zero-sublevel fix changes how existing ACL rules resolve. A rule written assensors/#previously did not match the topicsensors; it now does.If you rely on that distinction — for example a write-only rule
devices/#that must not grant read ondevicesitself — review your rules and, where needed, split them into an exact rule plus a wildcard rule. Re-run your rule set throughMatchTopicbefore upgrading if you are unsure.Helm chart
extraEnvpassthrough forCOMQTT_*overridesNote that
image.tagfalls back toChart.appVersion, so upgrading the chart without pinningimage.tagrolls pods onto the2.6.6image. Pinimage.tagbeforehand if you need to stage the rollout.Contributors
Thanks to everyone who contributed to this release:
Upgrade notes
ghcr.io/wind-c/comqtt:2.6.6(or the floatingv2.6tag).>= 2.6.0should note the raft snapshot restore fix: nodes that have already restored a broken snapshot are still running with an empty routing table and will stay that way until they re-subscribe or restart with the new binary.Full Changelog: https://github.com/wind-c/comqtt/compare/v2.6.5...v2.6.6
Configuration
📅 Schedule: (UTC)
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox.
🔕 Ignore: Close this MR and you won't be reminded about this update again.
This MR has been generated by Mend Renovate CLI.
mentioned in issue #1
changed title from
Update module github.com/wind-c/comqtt/v2 to v2.6.6toUpdate module github.com/wind-c/comqtt/v2 to v2.6.7changed the description
added 1 commit
c3bd9a79- Update module github.com/wind-c/comqtt/v2 to v2.6.7Compare with previous version
mentioned in commit
496f402d3d