[FR] Calculate mbpsSendRate and mbpsRecvRate for a Group

[maxsharabayko]

Calculate mbpsSendRate and mbpdRecvRate statistics for a group.
Update the docs, mainly API.statistics.md.

Dev Note

The group code fills in perf->mbpsSendRate and perf->mbpsRecvRate (see here), but perf->byteSent isn't updated anywhere. So those values stay zero.

[ethouris]

This is a bit problem. Indeed there are instructions filling these fields:

    perf->mbpsSendRate    = double(perf->byteSent) * 8.0 / interval;
    perf->mbpsRecvRate    = double(perf->byteRecv) * 8.0 / interval;

but perf-byteSent isn't updated anywhere, and so it likely stays with 0 value - and then so is the value of these two fields. The group collects transmission stats only through pktSentUnique and pktRecvUnique fields. The fields for byteSend and byteRecv are normally complete link usage stats and there's no found definition for them for a group. Not sure also what interpretation for mbpsSendRate would be useful because it doesn't make much sense for the group. For a single link it may mean how much the link is being burdened, but for groups it doesn't have any sensible interpretation - especially that in broadcast groups it would mean to burden all links equally, while in backup it would be almost equal to one link.

Actually I might be able to create a sensible interpretation for byteSent, which would be the number of bytes sent for a single group transmission over all links, including per-link retransmissions (would be hard to define anything similar for byteRecv, especially that in the version of common receiver buffer it would have to be done on the buffer level). But even if so, the send/recv rate stats based on these values would statistically make very little sense.

[maxsharabayko]

I think the simpler the better. A group should just show the rate of all its member links combined. Per-member rates can always be retrieved separately from each member.

[npanhaleux]

I though about it this week. I'm not sure I know every usages of group's mbpsSendRate but it could make sense to use the Min or the Max of members of this groups.

I mean, if it's to limite RTX for example, so use the min mbpsSendRate of all links associated to this group to ensure that the worst link won't have too many RTX.
If it's to compute a video order, use the min too to ensure the stream is able to be transmitted by every links in the group.
On another way, if it's to show statistics on graph, show the max of all members (as groups are used for bonding where the stream is duplicated, right ?)

If the usage of mbpsSendRate for groups is multiple, so why not get a minGroupMbpsSendRate and a maxGroupMbspSendRate and apply the right one depending on what we are asking for through readable methods:
getMaxGroupRtx() { return minGroupMbpsSendRate * maxRtxPercent
/ 100; };
getGraphValueGroup() { return maxGroupMbpsSendRate; };
getAvailableBdwGroup() { return minGroupMbpsSendRate; };

[ethouris]

We need to think about usefulness of such statistical data. What you described is something likely to be calculated differently and have a different meaning depending on the group type. Having something like min/max is when you think about the broadcast type, but with backup type you usually have only one link transmitting the data, while others transmit only the minimum keepalive packets in order to remain in standby. For balancing groups - which are not yet implemented, but we have always had them in plans, so they should be also taken into account here - it could be something even different, although this is the only type where collective value for all links together might make sense.

Also for the groups things look different here on the sender and the receiver. The worst case here is the broadcast-type group because only at the receiver side can you distinguish between the useful packets and the overhead packets; sender just sends and doesn't know whether the packet it has sent will be effectively unique or overhead. On the sender there exist the following possibilities for calculating anyhow interpreted sending rate:

• using unique sent bytes: represent the speed at which the current stream is being sent
• using total sent bytes: represent the overall transmission speed of everything, including overhead (of whatever kind - both redundant packets and uselessly retransmitted packets). Note here that collecting "all bytes sent" in the group is also not implemented.
• total sent bytes reduced appropriately per group, of course, only in broadcast-type group - that is: X = T - (U*(N-1)), where:
  • T : total packets
  • U : unique packets
  • N : number of links

The latter allows to use as a base of sent data the value that "would be" if the unique packets were sent only once, but include all overhead from retransmissions. For that, however, there are also no suitable fields in the stats structure and this value would have to be available only as an alternative.