Skip to content

Fix mixed types bug in pretty_print_dict() - #388

Open
oleole39 wants to merge 5 commits into
YunoHost:devfrom
oleole39:fix_print_pretty_dict_sorted
Open

oleole39 wants to merge 5 commits into
YunoHost:devfrom
oleole39:fix_print_pretty_dict_sorted

Conversation

@oleole39

@oleole39 oleole39 commented Oct 26, 2026 •

Copy link
Copy Markdown

Fixes YunoHost/issues#2910

sorted() crashes when keys are of mixed types so this PR fixes this by converting all types to strings when mixed types are detected.

  • This PR assumes that keys' original types would only be string, integer or float.

  • It uses additional module natsort instead of sorted() because it performs better at sorting mixed types (once converted to string) as show below tests:

    from natsort import natsorted 
    
    # str keys 
    >>> a = {"apricot": "test", "apple": "test2", "pear": "test3"}
    >>> sorted(a.keys())
    ['apple', 'apricot', 'pear']
    >>> natsorted(a.keys())
    ['apple', 'apricot', 'pear']
    
    # int keys
    >>> b = {1: "test", 3: "test2", 2: "test3"}
    >>> sorted(b.keys())
    [1, 2, 3]
    >>> natsorted(b.keys())
    [1, 2, 3]
    
    # float keys
    >>> c = {1.398: "test", 1.38: "test2", 1.3729: "test3"}
    >>> sorted(c.keys())
    [1.3729, 1.38, 1.398]
    >>> natsorted(c.keys())
    [1.3729, 1.38, 1.398]
    
    # port numbers with ranges (after conversion to strings)
    >>> d = {"1233":"test", "1838":"test2", "1234-1837":"test3"}
    >>> sorted(d.keys())
    ['1233', '1234-1837', '1838']
    >>> natsorted(d.keys())
    ['1233', '1234-1837', '1838']
    
    # mixed types (after conversion to strings)
    >>> e = {"134":"test", "1234-1837":"test2", "777":"test3", "hello":"test3", "9999":"test4", "124.213":"test5"}
    >>> sorted(e.keys())
    ['1234-1837', '124.213', '134', '777', '9999', 'hello'] #literal sort: 123(...) comes before 124(...)
    >>> natsorted(e.keys())
    ['124.213', '134', '777', '1234-1837', '9999', 'hello']

These two lines are optional since it should also work well when systematically converting keys to string type. But performances might be slightly lower (untested) ?

@oleole39

oleole39 commented Oct 27, 2026 •

Copy link
Copy Markdown
Author

Ran some perf tests

perftest.py
import time

a = {"apricot": "test", "apple": "test2", "pear": "test3"}
b = {1: "test", 3: "test2", 2: "test3"}
c = {1.398: "test", 1.38: "test2", 1.3729: "test3"}
d = {"1233":"test", "1838":"test2", "1234-1837":"test3"}
e = {134:"test", "1234-1837":"test2", 777:"test3", "hello":"test3", 9999:"test4", 124.213:"test5"}
test_dicts = [a, b, c, d, e]

start = time.time()
from natsort import natsorted
end = time.time()
exec_time = end - start
print(f'0) {exec_time:.7f}s importing `natsort` package')

# Check types before converting all types to string, only use sorted
start = time.time()
for test_dict in test_dicts:
  keys = test_dict.keys()
  keys_types = set(type(k) for k in keys)
  if len(keys_types) > 1:
    keys = [str(k) for k in keys] 
  keys = sorted(keys)
end = time.time()
exec_time = end - start
print(f'1) {exec_time:.7f}s string conversion only when mixed types, use only sorted')

# Check types before converting all types to string, use natsorted only when mixed types
start = time.time()
for test_dict in test_dicts:
  keys = test_dict.keys()
  keys_types = set(type(k) for k in keys)
  if len(keys_types) > 1:
    keys = [str(k) for k in keys] 
    keys = natsorted(keys)
  else:
    keys = sorted(keys)
end = time.time()
exec_time = end - start
print(f'2) {exec_time:.7f}s string conversion only when mixed types, use natsorted if so else use sorted')

# Check types before converting all types to string, use only natsorted
start = time.time()
for test_dict in test_dicts:
  keys = test_dict.keys()
  keys_types = set(type(k) for k in keys)
  if len(keys_types) > 1:
    keys = [str(k) for k in keys] 
  keys = natsorted(keys)
end = time.time()
exec_time = end - start
print(f'3) {exec_time:.7f}s string conversion only when mixed types, use only natsorted')

# Systematically convert types to strings, use only sorted
start = time.time()
for test_dict in test_dicts:
  keys = test_dict.keys()
  keys = [str(k) for k in keys] 
  keys = sorted(keys)
end = time.time()
exec_time = end - start
print(f'4) {exec_time:.7f}s always convert types to strings, use only sorted')

# Systematically convert types to strings, use only natsorted
start = time.time()
for test_dict in test_dicts:
  keys = test_dict.keys()
  keys = [str(k) for k in keys] 
  keys = natsorted(keys)
end = time.time()
exec_time = end - start
print(f'5) {exec_time:.7f}s always convert types to strings, use only natsorted')
(venv) me@computer:~$ python perftest.py
0) 0.0552204s importing `natsort` package
1) 0.0000339s string conversion only when mixed types, use only sorted
2) 0.0002084s string conversion only when mixed types, use natsorted if so else use sorted
3) 0.0007517s string conversion only when mixed types, use only natsorted
4) 0.0000184s always convert types to strings, use only sorted
5) 0.0005391s always convert types to strings, use only natsorted

So performance-wise, I'd recommend option 1 (let's not convert types when not mixed, as it might avoid literal sort behaviors I have not thought of with sorted()).
Or if we want better mixed types sorting and that we don't mind adding one pip package, go for option 2 (~10x slower, although we remain in the sub-millisecond scale), to which must also be added once the time for importing the module (~1600x slower).

@oleole39
oleole39 force-pushed the fix_print_pretty_dict_sorted branch from 79d86d1 to e832f15 Compare October 27, 2026 14:15
@oleole39
oleole39 force-pushed the fix_print_pretty_dict_sorted branch from e832f15 to 18a0363 Compare October 27, 2026 14:16
@oleole39

oleole39 commented Oct 27, 2026 •

Copy link
Copy Markdown
Author

I am more comfortable with natsort due to the fact I don't have in mind all possible data supplied to pretty_print_dict() but it costs a bit performance-wise...

Here is an alternative approach to natsort by matching specific case (e.g. port-range) via regex and fixing the sorting function on a case by case basis.

It therefore works well with dict e (fixed scenario) but would fail with dict f and trigger a MoulinetteError.

e = {134:"test", "1234-1837":"test2", 777:"test3", 9999:"test4", 1233: "test5"}
f = {134:"test", "1234-1837":"test2", 777:"test3", "hello":"test3", 9999:"test4", 124.213:"test5", 1233: "test6"}
perftest.py
import time

a = {"apricot": "test", "apple": "test2", "pear": "test3"}
b = {1: "test", 3: "test2", 2: "test3"}
c = {1.398: "test", 1.38: "test2", 1.3729: "test3"}
d = {"1233":"test", "1838":"test2", "1234-1837":"test3"}
e = {134:"test", "1234-1837":"test2", 777:"test3", 9999:"test4", 1233: "test5"}
f = {134:"test", "1234-1837":"test2", 777:"test3", "hello":"test3", 9999:"test4", 124.213:"test5", 1233: "test6"}
test_dicts = [a, b, c, d, e, f]

start = time.time()
from natsort import natsorted
end = time.time()
exec_time = end - start
print(f'0) {exec_time:.7f}s importing `natsort` package')

start = time.time()
import re
end = time.time()
exec_time = end - start
print(f'0) {exec_time:.7f}s importing `re` package')

# Check types before converting all types to string, only use sorted
start = time.time()
for test_dict in test_dicts:
  keys = test_dict.keys()
  keys_types = set(type(k) for k in keys)
  if len(keys_types) > 1:
    keys = [str(k) for k in keys]
  keys = sorted(keys)
end = time.time()
exec_time = end - start
print(f'1) {exec_time:.7f}s string conversion only when mixed types, use only sorted')

# Check types before converting all types to string, use natsorted only when mixed types
start = time.time()
for test_dict in test_dicts:
  keys = test_dict.keys()
  keys_types = set(type(k) for k in keys)
  if len(keys_types) > 1:
    keys = [str(k) for k in keys]
    keys = natsorted(keys)
  else:
    keys = sorted(keys)
end = time.time()
exec_time = end - start
print(f'2) {exec_time:.7f}s string conversion only when mixed types, use natsorted if so else use sorted')

# Check types before converting all types to string, use only natsorted
start = time.time()
for test_dict in test_dicts:
  keys = test_dict.keys()
  keys_types = set(type(k) for k in keys)
  if len(keys_types) > 1:
    keys = [str(k) for k in keys]
  keys = natsorted(keys)
end = time.time()
exec_time = end - start
print(f'3) {exec_time:.7f}s string conversion only when mixed types, use only natsorted')

# Systematically convert types to strings, use only sorted
start = time.time()
for test_dict in test_dicts:
  keys = test_dict.keys()
  keys = [str(k) for k in keys]
  keys = sorted(keys)
end = time.time()
exec_time = end - start
print(f'4) {exec_time:.7f}s always convert types to strings, use only sorted')

# Systematically convert types to strings, use only natsorted
start = time.time()
for test_dict in test_dicts:
  keys = test_dict.keys()
  keys = [str(k) for k in keys]
  keys = natsorted(keys)
end = time.time()
exec_time = end - start
print(f'5) {exec_time:.7f}s always convert types to strings, use only natsorted')

# Check types before converting all types to string, use regex (re.sub) to manage port range case
start = time.time()
for test_dict in test_dicts:
    keys = list(test_dict.keys())
    keys_types = set(type(k) for k in keys)
    # Keys are assumed to be of integer, float or/and string type
    if len(keys_types) > 1:
        #import re
        replacements = {}
        # For ports range (e.g. "150-194") case, sort using only start value
        for k in keys:
            if isinstance(k, str) and re.search(r"^([0-9]+)-[0-9]+$", k):
                keys.remove(k)
                k_sort = int(re.sub(r"^([0-9]+)-[0-9]+$",r"\1", k))
                keys.append(k_sort)
                replacements[k_sort] = k
        # Make sure all keys are now of the same type
        keys_types = set(type(k) for k in keys)
        if len(keys_types) > 1:
            #raise MoulinetteError(f"Can't prettify dictionnary: one or more key are of an unexpected type", raw_msg=True)
            print(f'MoulinetteError for {test_dict}')
        else:
            keys = sorted(keys)
            keys = [replacements[k] if k in replacements else k for k in keys]
    else:
        keys = sorted(keys)
end = time.time()
exec_time = end - start
print(f'6) {exec_time:.7f}s string conversion only when mixed types, manage port range case with re.sub')

# Check types before converting all types to string, use regex (re.search) and split() to manage port range case
start = time.time()
for test_dict in test_dicts:
    keys = list(test_dict.keys())
    keys_types = set(type(k) for k in keys)
    # Keys are assumed to be of integer, float or/and string type
    if len(keys_types) > 1:
        #import re
        replacements = {}
        # For ports range (e.g. "150-194") case, sort using only start value
        for k in keys:
            if isinstance(k, str) and re.search(r"^([0-9]+)-[0-9]+$", k):
                k_sort = int(k.split("-")[0])
                keys.remove(k)
                replacements[k_sort] = k
                keys.append(k_sort)
        # Make sure all keys are now of the same type
        keys_types = set(type(k) for k in keys)
        if len(keys_types) > 1:
            #raise MoulinetteError(f"Can't prettify dictionnary: one or more key are of an unexpected type", raw_msg=True)
            print(f'MoulinetteError for {test_dict}')
        else:
            keys = sorted(keys)
            keys = [replacements[k] if k in replacements else k for k in keys]
    else:
        keys = sorted(keys)
end = time.time()
exec_time = end - start
print(f'7) {exec_time:.7f}s string conversion only when mixed types, manage port range case with re.search + split')
(venv) me@computer:~$ python perftest.py
0) 0.0514917s importing `natsort` package
0) 0.0000031s importing `re` package
1) 0.0000491s string conversion only when mixed types, use only sorted
2) 0.0004523s string conversion only when mixed types, use natsorted if so else use sorted
3) 0.0009241s string conversion only when mixed types, use only natsorted
4) 0.0000231s always convert types to strings, use only sorted
5) 0.0006435s always convert types to strings, use only natsorted
MoulinetteError for {134: 'test', '1234-1837': 'test2', 777: 'test3', 'hello': 'test3', 9999: 'test4', 124.213: 'test5', 1233: 'test6'}
6) 0.0002902s string conversion only when mixed types, manage port range case with re.sub
MoulinetteError for {134: 'test', '1234-1837': 'test2', 777: 'test3', 'hello': 'test3', 9999: 'test4', 124.213: 'test5', 1233: 'test6'}
7) 0.0000591s string conversion only when mixed types, manage port range case with re.search + split

Note that time does not seem so accurate in these tests - in particular switching the order of 6) and 7) also almost reverse their respective performance cost... maybe due to some caching?
Similarly, switching the order of import instructions changes significantly the perf data:

0) 0.0079355s importing `re` package
0) 0.0249119s importing `natsort` package

@Josue-T

Josue-T commented Oct 27, 2026

Copy link
Copy Markdown
Contributor

Hello,

Thanks for the PR and this important fix. In my opinion I would prefer to not add natsort as the dependency and try to only use the native sort function, because this lib seem not well maintained, the last releases seem to be 3 year old. So even if the perf is a bit lower I prefer to use the native Python function. But it's still my opinion, let's see maybe what other people think about this.

@oleole39

Copy link
Copy Markdown
Author

So even if the perf is a bit lower I prefer to use the native Python function.

Perf actually seems to be better with native Python, but it seems more likely to break when facing new unforeseen cases.

I would prefer to not add natsort [...] because this lib seem not well maintained

To be fair regarding the lib's maintenance history, there seems to be new releases on the way: https://github.com/SethMMorton/natsort/pulls
But no strong opinion on my side.

Comment thread moulinette/interfaces/cli.py Outdated
Comment on lines +208 to +212
keys = [str(k) for k in keys]
# `natsorted()` does proper numeric sorting when dealing with
# numbers even when the data is of string type
from natsort import natsorted
keys = natsorted(keys)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In my opinion we can do just something like this:

Suggested change
keys = [str(k) for k in keys]
# `natsorted()` does proper numeric sorting when dealing with
# numbers even when the data is of string type
from natsort import natsorted
keys = natsorted(keys)
keys = sorted(keys, key=lambda k: str(k))

@oleole39 oleole39 Oct 28, 2026 •

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is enough to fix the bug, however the sort becomes literal and it would look weird in various cases, like typically in the linked issue (firewall port list), e.g.:

{134:"test", "1234-1837":"test2", 111111:"test3", 9999:"test4", 1233: "test5"}

will become:

[111111, 1233, '1234-1837', 134, 9999]

instead of expected:

[134, 1233, '1234-1837', 9999, 111111]

This is basically the same approach as in my test 4) above, even if the use of lambda function is probably better (saves ~20 tens of microsecond!).

I solved this issue for this specific case in tests 6 and 7 (with the help of additional re module, but that's almost native python I guess), cf. second perftest.py (in folded code block) above.

@zamentur

Copy link
Copy Markdown
Member

Note that i am not totally sure that yunohost firewall support 1234-1837 notation !

@oleole39

Copy link
Copy Markdown
Author

i am not totally sure that yunohost firewall support 1234-1837 notation !

I ran the regen-conf code using the firewall.yml provided in the issue linked to this PR, and below is the beginning of the generated yunohost-firewall.conf:

#!/usr/sbin/nft -f

define tcp_ports = { 25, 80, 443, 587, 993, 4190, 5222, 5223, 5269, 5349, 5350, 5352, 5353, 7719 }

define udp_ports = { 3478, 3479, 5349, 5350, 5352, 5353, 49152-65535, 1194 }


table inet filter {
    chain input {
        policy drop;
        ct state related,established counter accept;

        tcp dport $tcp_ports counter accept;
        
        udp dport $udp_ports counter accept;

And port ranges indicated with a dash do seem supported in nftables, cf.:

@Josue-T

Josue-T commented Oct 29, 2026

Copy link
Copy Markdown
Contributor

We are using the notation like yunohost firewall allow Both 49153:49193 since a long time. It's documente like here.

@oleole39

oleole39 commented Oct 29, 2026 •

Copy link
Copy Markdown
Author

We are using the notation like yunohost firewall allow Both 49153:49193 since a long time.

On the UI side only though since migration to nftables, apparently due to some iptables legacy:
https://github.com/YunoHost/yunohost/blob/15e88bd21170c657160f276394f8cda25353dfec/src/firewall.py#L114-L115
So I assume it is saved as 49153-49193 in firewall.yml from which regen-conf hook then generates the nftables config.
As for pretty_print_dict(), I believe the data direction goes only from the config file as input to the UI as output (haven't traced the opposite direction though)

@Josue-T

Josue-T commented Oct 29, 2026

Copy link
Copy Markdown
Contributor

you are right, it's legacy stuff that would need to be updated.

Comment thread moulinette/interfaces/cli.py Resolved
@oleole39

oleole39 commented Nov 13, 2026 •

Copy link
Copy Markdown
Author

To summarize, there is a decision left to take here with mainly 3 choices:

  1. use natsorted() only when facing mixed types (labelled option 2 in tests) - simple and most flexible solution, but slower and relying on an additonal module which is mostly maintained by a single person. This is the option implemented so far in this PR if you review the commits.
  2. systematically convert values to strings and use sorted() (labelled option 4 in tests) - simple and most performing, no additional module needed, but the output will not appear sorted correctly in some cases (typically when there is a port range in the list) due to litteral sort.
    See second perftest.py folded code block to check the code of mentioned options.
  3. convert to strings only when facing mixed types, then manage cases specifically with re module (only port range case is implemented so far as currently no other case in mind) (labelled option 6 or 7 in tests - one uses re.sub() the other split(), not clear which one is performing better). Longer code, but best tradeoff between performance cost, good handling of the firewall port range case, and additional module (relies on re module which is very common).
    See second perftest.py folded code block to check the code of mentioned options. Note that the re.search() check is here as it is better performance-wise than checking systematically all values.

Co-authored-by: zamentur <4080016+zamentur@users.noreply.github.com>
Co-authored-by: sitegui <2801575+sitegui@users.noreply.github.com>
@oleole39

Copy link
Copy Markdown
Author

@josue @Salamandar
CC: @ljf @sitegui

In the contributors' meeting, we've been thinking to keep natural sort order, but using a much simpler sorting function than natsort module (cf. 2246981). It only requires loading re module and seems to work as well as natsort for our need.
Overall, it's like 10x slower than sorted() but we talk of sub-millisecond scale, and offer a better UX.

Sorting the following sample:
   [124, '1113-1837', 777, 'hello', 999.98, 999.94, '1.2.3~yn4', '1.2.3~yn1', '1.2.3~yn12', '1.2.1~yn15', 'dom2', 'dom15', 'DOM1']
0) 0.0485466s importing `natsort` package
0) 0.0000012s importing `re` package
1) string conversion only when mixed types, use only sorted
   ['1.2.1~yn15', '1.2.3~yn1', '1.2.3~yn12', '1.2.3~yn4', '1113-1837', '124', '777', '999.94', '999.98', 'DOM1', 'dom15', 'dom2', 'hello']
0.0000246s
2) string conversion only when mixed types, use natsorted if so else use sorted
   ['1.2.1~yn15', '1.2.3~yn1', '1.2.3~yn4', '1.2.3~yn12', '124', '777', '999.94', '999.98', '1113-1837', 'DOM1', 'dom2', 'dom15', 'hello']
0.0002000s
3) string conversion only when mixed types, use only natsorted
   ['1.2.1~yn15', '1.2.3~yn1', '1.2.3~yn4', '1.2.3~yn12', '124', '777', '999.94', '999.98', '1113-1837', 'DOM1', 'dom2', 'dom15', 'hello']
0.0001700s
4) always convert types to strings, use only sorted
   ['1.2.1~yn15', '1.2.3~yn1', '1.2.3~yn12', '1.2.3~yn4', '1113-1837', '124', '777', '999.94', '999.98', 'DOM1', 'dom15', 'dom2', 'hello']
0.0000148s
5) always convert types to strings, use only natsorted
   ['1.2.1~yn15', '1.2.3~yn1', '1.2.3~yn4', '1.2.3~yn12', '124', '777', '999.94', '999.98', '1113-1837', 'DOM1', 'dom2', 'dom15', 'hello']
0.0001364s
MoulinetteError for {124: 'test', '1113-1837': 'test2', 777: 'test3', 'hello': 'test3', 999.98: 'test4', 999.94: 'test5', '1.2.3~yn4': 'test6', '1.2.3~yn1': 'test7', '1.2.3~yn12': 'test8', '1.2.1~yn15': 'test9', 'dom2': 'test10', 'dom15': 'test11', 'DOM1': 'test12'}
6) string conversion only when mixed types, manage port range case with re.sub
   [124, 777, 'hello', 999.98, 999.94, '1.2.3~yn4', '1.2.3~yn1', '1.2.3~yn12', '1.2.1~yn15', 'dom2', 'dom15', 'DOM1', 1113]
0.0001557s
MoulinetteError for {124: 'test', '1113-1837': 'test2', 777: 'test3', 'hello': 'test3', 999.98: 'test4', 999.94: 'test5', '1.2.3~yn4': 'test6', '1.2.3~yn1': 'test7', '1.2.3~yn12': 'test8', '1.2.1~yn15': 'test9', 'dom2': 'test10', 'dom15': 'test11', 'DOM1': 'test12'}
7) string conversion only when mixed types, manage port range case with re.search + split
   [124, 777, 'hello', 999.98, 999.94, '1.2.3~yn4', '1.2.3~yn1', '1.2.3~yn12', '1.2.1~yn15', 'dom2', 'dom15', 'DOM1', 1113]
0.0000572s
8) always convert types to strings, use only custom natural_sort
   ['1.2.1~yn15', '1.2.3~yn1', '1.2.3~yn4', '1.2.3~yn12', 124, 777, 999.94, 999.98, '1113-1837', 'DOM1', 'dom2', 'dom15', 'hello']
0.0001249s
perftest.py
import time

a = {"apricot": "test", "apple": "test2", "pear": "test3"}
b = {1: "test", 3: "test2", 2: "test3"}
c = {1.398: "test", 1.38: "test2", 1.3729: "test3"}
d = {"1233":"test", "1838":"test2", "1234-1837":"test3"}
e = {134:"test", "1234-1837":"test2", 777:"test3", 9999:"test4", 1233: "test5"}
f = {134:"test", "1234-1837":"test2", 777:"test3", "hello":"test3", 9999:"test4", 124.213:"test5", 1233: "test6"}
g = {124:"test", "1113-1837":"test2", 777:"test3", "hello":"test3", 999.98:"test4", 999.94:"test5", "1.2.3~yn4": "test6", "1.2.3~yn1": "test7", "1.2.3~yn12": "test8", "1.2.1~yn15": "test9", "dom2": "test10", "dom15": "test11", "DOM1": "test12"}

#test_dicts = [a, b, c, d, e, f, g]
test_dicts = [g]
print('Sorting the following sample:')
for test_dict in test_dicts:
    print ("   " + str(list(test_dict.keys())))

start = time.time()
from natsort import natsorted
end = time.time()
exec_time = end - start
print(f'0) {exec_time:.7f}s importing `natsort` package')

start = time.time()
import re
end = time.time()
exec_time = end - start
print(f'0) {exec_time:.7f}s importing `re` package')

# 1. Check types before converting all types to string, only use sorted
start = time.time()
for i, test_dict in enumerate(test_dicts):
  keys = test_dict.keys()
  keys_types = set(type(k) for k in keys)
  if len(keys_types) > 1:
    keys = [str(k) for k in keys]
  keys = sorted(keys)
  print(f'1) string conversion only when mixed types, use only sorted')
  print("   " + str(keys)) if test_dict is g else ''
end = time.time()
exec_time = end - start
print(f'{exec_time:.7f}s')

# 2. Check types before converting all types to string, use natsorted only when mixed types
start = time.time()
for i, test_dict in enumerate(test_dicts):
  keys = test_dict.keys()
  keys_types = set(type(k) for k in keys)
  if len(keys_types) > 1:
    keys = [str(k) for k in keys]
    keys = natsorted(keys)
  else:
    keys = sorted(keys)
  if i == 0:
    print(f'2) string conversion only when mixed types, use natsorted if so else use sorted')
  print("   " + str(keys))
end = time.time()
exec_time = end - start
print(f'{exec_time:.7f}s')

# 3. Check types before converting all types to string, use only natsorted
start = time.time()
for i, test_dict in enumerate(test_dicts):
  keys = test_dict.keys()
  keys_types = set(type(k) for k in keys)
  if len(keys_types) > 1:
    keys = [str(k) for k in keys]
  keys = natsorted(keys)
  if i == 0:
    print(f'3) string conversion only when mixed types, use only natsorted')
  print("   " + str(keys))
end = time.time()
exec_time = end - start
print(f'{exec_time:.7f}s')

# 4. Systematically convert types to strings, use only sorted
start = time.time()
for i, test_dict in enumerate(test_dicts):
  keys = test_dict.keys()
  keys = [str(k) for k in keys]
  keys = sorted(keys)
  if i == 0:
    print(f'4) always convert types to strings, use only sorted')
  print("   " + str(keys))
end = time.time()
exec_time = end - start
print(f'{exec_time:.7f}s')

# 5. Systematically convert types to strings, use only natsorted
start = time.time()
for i, test_dict in enumerate(test_dicts):
  keys = test_dict.keys()
  keys = [str(k) for k in keys]
  keys = natsorted(keys)
  if i == 0:
    print(f'5) always convert types to strings, use only natsorted')
  print("   " + str(keys))
end = time.time()
exec_time = end - start
print(f'{exec_time:.7f}s')

# 6. Check types before converting all types to string, use regex (re.sub) to manage port range case
start = time.time()
for i, test_dict in enumerate(test_dicts):
    keys = list(test_dict.keys())
    keys_types = set(type(k) for k in keys)
    # Keys are assumed to be of integer, float or/and string type
    if len(keys_types) > 1:
        #import re
        replacements = {}
        # For ports range (e.g. "150-194") case, sort using only start value
        for k in keys:
            if isinstance(k, str) and re.search(r"^([0-9]+)-[0-9]+$", k):
                keys.remove(k)
                k_sort = int(re.sub(r"^([0-9]+)-[0-9]+$",r"\1", k))
                keys.append(k_sort)
                replacements[k_sort] = k
        # Make sure all keys are now of the same type
        keys_types = set(type(k) for k in keys)
        if len(keys_types) > 1:
            #raise MoulinetteError(f"Can't prettify dictionnary: one or more key are of an unexpected type", raw_msg=True)
            print(f'MoulinetteError for {test_dict}')
        else:
            keys = sorted(keys)
            keys = [replacements[k] if k in replacements else k for k in keys]
    else:
        keys = sorted(keys)
    if i == 0:
        print(f'6) string conversion only when mixed types, manage port range case with re.sub')
    print("   " + str(keys))
end = time.time()
exec_time = end - start
print(f'{exec_time:.7f}s')

# 7. Check types before converting all types to string, use regex (re.search) and split() to manage port range case
start = time.time()
for i, test_dict in enumerate(test_dicts):
    keys = list(test_dict.keys())
    keys_types = set(type(k) for k in keys)
    # Keys are assumed to be of integer, float or/and string type
    if len(keys_types) > 1:
        #import re
        replacements = {}
        # For ports range (e.g. "150-194") case, sort using only start value
        for k in keys:
            if isinstance(k, str) and re.search(r"^([0-9]+)-[0-9]+$", k):
                k_sort = int(k.split("-")[0])
                keys.remove(k)
                replacements[k_sort] = k
                keys.append(k_sort)
        # Make sure all keys are now of the same type
        keys_types = set(type(k) for k in keys)
        if len(keys_types) > 1:
            #raise MoulinetteError(f"Can't prettify dictionnary: one or more key are of an unexpected type", raw_msg=True)
            print(f'MoulinetteError for {test_dict}')
        else:
            keys = sorted(keys)
            keys = [replacements[k] if k in replacements else k for k in keys]
    else:
        keys = sorted(keys)
    if i == 0:
      print(f'7) string conversion only when mixed types, manage port range case with re.search + split')
    print("   " + str(keys))
end = time.time()
exec_time = end - start
print(f'{exec_time:.7f}s')

# 8. Systematically convert types to strings, use only custom natural_sort
start = time.time()
def natural_sort(l): 
    # borrowed from https://stackoverflow.com/questions/4836710/is-there-a-built-in-function-for-string-natural-sort/4836734#4836734
	convert = lambda text: int(text) if text.isdigit() else text.lower()
	alphanum_key = lambda key: [convert(c) for c in re.split('([0-9]+)', str(key))]
	return sorted(l, key=alphanum_key)
for i, test_dict in enumerate(test_dicts):
  keys = test_dict.keys()
  keys = natural_sort(keys)
  if i == 0:
    print(f'8) always convert types to strings, use only custom natural_sort')
  print("   " + str(keys))
end = time.time()
exec_time = end - start
print(f'{exec_time:.7f}s')

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

yunohost firewall list -r Python error

4 participants