create single bytes instance from sequence of memoryview

Question

tl;dr Given a Sequence of memoryview, how can I create a single bytes instance without creating intermediate bytes instances?

The naive approach creates many intermediary instances of bytes

def create_bytes(seq_mv: Sequence[memoryview]) -> bytes:
    data = bytes()
    for mv in seq_mv:
        data = data + bytes(mv)
    return data

The function create_bytes creates len(seq_mv) + 1 instances of bytes during execution. That is inefficient.
I want create_bytes to create one new bytes instance during execution.

Did you try bytes().join(seq_mv)?

Tim Peters
– Tim Peters

2020-10-11 04:24:21 +00:00
Commented Oct 11, 2020 at 4:24 — Tim Peters
– Tim Peters, Commented Oct 11, 2020 at 4:24
Indeed, join is here in the docs, too. Thanks!

JamesThomasMoon
– JamesThomasMoon

2020-10-11 04:50:59 +00:00
Commented Oct 11, 2020 at 4:50 — JamesThomasMoon
– JamesThomasMoon, Commented Oct 11, 2020 at 4:50

jsbueno · Accepted Answer · 2020-10-11 04:30:25Z

4

bytes as you got it, is an imutable object.

As the Tim Peters put in the comments, you can let Python create a single instance with all parts joined together with a single call to bytes().join(seq_mv).

If you need to perform any other operation on your data that would involve changing it in the way, you could be using the mutable bytearray instead- which not only gives you flexibility to change your object, but have all the advantages of mutable sequences.

You can then make a single conversion to bytes at the end of your function if the users of your function can't deal straight with a a bytearray (but maybe you can just return it directly):

def create_bytes(seq_mv: Sequence[memoryview]) -> bytes:
    data = bytearray()
    for mv in seq_mv:
        data.extend(mv)
    return bytes(data)

Or simply:

from functools import reduce

data = reduce(lambda data, mv: data.extend(mv), seq_mv, bytearray())

edited Oct 11, 2020 at 4:30

answered Oct 11, 2020 at 4:23

jsbueno

114k11 gold badges159 silver badges239 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

JamesThomasMoon Over a year ago

Ah, good use of reduce! This is a nice extended answer ;-) . Indeed, join here in the docs. Thanks!

Collectives™ on Stack Overflow

create single bytes instance from sequence of memoryview

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related