Questions tagged [data-structures]
Questions regarding efficient storage and representation of data within a software application.
919 questions
-1
votes
4
answers
206
views
Do GML switch-case statements perform worse than function tables?
The reason I wanted to ask is because of some code from Undertale that is responsible for choosing which dialogue set to use. It works something like this:
switch (id) {
case 0:
msg[0] = &...
- 665
5
votes
7
answers
550
views
Recommended data structures/algorithms for checking peoples' availability schedules
I'm working on a platform that allows assigning users to events manually. Every user provides their general availability (Mondays 2PM - 8PM, Tuesdays not at all, Wednesdays 3:30PM-7PM, and so on). ...
- 159
6
votes
6
answers
903
views
Good way to represent connectivity between nodes of tree (outside of tree hierarchy)?
I have a set of data carrier objects represented using vertices in a strict tree hierarchy. I also have the following requirements:
some vertices should be "linked" to other vertices ...
- 69
0
votes
2
answers
162
views
Designing a social media notification system, would a single table with JSON column be the correct approach for data?
I am trying to create a notification system similar to Instagram, for the lack of better comparasion.
The Goal:
Getting user notifications in a unify way, for easy unparsing and displaying to user.
...
- 101
0
votes
1
answer
166
views
Generalised indexing scheme for variable memory layouts
structs are great, the only problem is that not only do they associate data (topo)logically, they also crystallise a memory layout for that set of concepts. I would like to avoid the latter.
So I'm ...
- 781
2
votes
3
answers
194
views
Domain data classes with fluid property lists
Is it fine for your data models to have a sort of "open-ended" list of properties?
For example, you may have some Employee:
// getters, setters, annotations are omitted
public class Employee ...
- 2,030
4
votes
1
answer
269
views
How do I handle polymorphic domain models in a Clean Architecture data access layer?
Let's say I'm making a space 4X game (because I am!) and I have an ISpaceObject interface in my domain layer, with a number of classes such as Ship, Starbase, Planet, Wormhole, etc. implementing it. I ...
- 577
0
votes
1
answer
170
views
Reorganize data with pattern while migrating from SQL to MongoDB
I have several tables in Oracle, in each of them are stored data about a different kind of tax declaration (house tax, hotel tax, and so on). All of them have some data in common, so I want to migrate ...
- 531
0
votes
2
answers
237
views
Leveraging Qt models for nested data structures
Here is a simplified version of what I'm trying to achieve. For every Store that is added, one or more Departments can be added. For every Department that is added, one or more Employees can be added. ...
- 133
-2
votes
2
answers
141
views
Map value to set of indexes [closed]
Assuming you have a Map and Set type, you can construct this structure.
type IndexMap<T> = Map<T, Set<int>>
This would be a Map (or dictionary) where the key is the generic type T ...
2
votes
0
answers
108
views
Efficient FIFO Data Storage and Deletion for Sequential Data [closed]
I’m building a device that needs to store data sequentially, retrieve it in chunks using FIFO, and then delete it. My device uses the ESP32-S3, and I'm currently using an SQLite build. However, as the ...
- 129
-1
votes
2
answers
294
views
SQLite - Design use of Variable Length Integers - explain the design flaw
The Carnegie Mellon Database Group invited the creator of sqlite D. Richard Hipp to talk about SQLite - The Databaseology Lectures - CMU Fall 2015.
Richard Hipp shows this slide (~ 39m51s) and says ...
- 183
2
votes
3
answers
1k
views
Is breaking encapsulation a necessary compromise for serialization?
I've been considering the way to solve this problem for a while, and I'm currently stuck between two options I both feel are suboptimal. Here's an abstract example of what I'm dealing with:
I have a ...
- 39
2
votes
2
answers
135
views
How to split database entities or access responsibilities of services to these entities
I have two services, auth-sync and twitter-indexer.
We use Privy for auth (incl. SSO with 3rd parties). We sync our userbase from Privy to our own DB for certain analytics purposes. And the auth-sync ...
- 191
-1
votes
2
answers
317
views
Are there any standards for storing checksums of a repository?
I have a repository with many files (mostly binary, images and raw data) and some documentation. The files are stored in a hierarchical folder structure; I want to allow checking the fixity of the ...
- 273
-1
votes
2
answers
491
views
Text files vs. MySQL Performance for Large Dataset
I am currently making a system for users to generate flashcards for Languages.
This involves adding information such as the definition, pronunciation, and example sentences for a word.
So far, I have ...
-2
votes
1
answer
122
views
Advice on data structure for an analysis and visualisation tool (javascript)
I am writing a browser-based tool to manipulate and visualise data (with D3.js). Currently, I store data in a JSON format, where each table is an object and columns are arrays. eg:
{
"data&...
- 97
-1
votes
1
answer
98
views
Designing a Distributed System for Indigenous Data Sovereignty Across Nations [closed]
I'm looking for some quick "back-of-the-napkin" thoughts from systems engineers on the following scenario:
A collection of Indigenous groups (in the US, Canada, and Australia) want to design ...
3
votes
3
answers
466
views
Encapsulating a central soap service in a microservice?
I am currently developing an application (rest-backend and SPA) that requires information from people (personal data, department, superior... ). Our company has a central directory for this purpose. ...
- 77
1
vote
1
answer
114
views
How to reuse parts of a Trie data structure (like a Trie DAG)?
I am playing around with a cross-language spell-check sort of thing, and am still in the prototyping/ideation phases. Basically I have thought of something like a Trie data structure, but I keep ...
- 2,787
0
votes
1
answer
257
views
Elegant way in C to store many parameters with default value and current value in embedded flash
I'm programming an embedded system that has a number of user configurable parameters, which are stored in flash memory. I have to store a default value for each parameter as well as the user settings. ...
- 175
0
votes
1
answer
832
views
Best practice - depending on object key iteration order?
In Javascript, the iteration order of an Object is enforced since ES2020 (or before, depending...).
However, is it a good idea (i.e. best practice) to depend on this?
I'm currently working on a ...
- 733
1
vote
0
answers
473
views
Implementing a memory efficient Abstract Syntax Tree
I am writing a compiler in C++ 20. I am looking for ways to improve heap memory performance. For example, I can compile a 36.8 MB source file that is just many repeating lines of:
let x0: string = &...
- 19
2
votes
1
answer
224
views
Wondering about ADT abstract data classes in Python
I am trying to learn a little Python by myself and came accross this book Data Structures and Algorithms Using Python Rance D. Necaise 2011 , to get some proper/formal/correct definitions/terminology ...
- 139
3
votes
4
answers
2k
views
Representing vectors as arrays of points vs. as data structures
I'm writing a program in Java where I need to represent the position, scale, and other 3-dimensional properties of objects in a world using vectors.
I can use either of these two approaches:
...
- 665
-2
votes
1
answer
325
views
Making objects aware of their (consistent) IDs
Abstracted problem
I would like to be able to quickly refer to some (most likely non-hashable) objects. At the same time, I would like them to be aware of their own identity. The 2-way search should ...
- 7
5
votes
9
answers
3k
views
Methods to increase the amount of data sent in a packet
I have been working on launching high-altitude balloons (HABs, or weather balloons) and I have been using LoRa to enable long-range communication with my balloons. It's been great and pretty reliable, ...
- 75
1
vote
1
answer
747
views
Tagged pointers vs. fat pointers
I'm writing my own dynamic programming language. So far I've been using a tagged union for all values. Since this boxes primitive types needlessly, I've begun researching tagged pointers which seem to ...
- 111
0
votes
0
answers
113
views
What is the Big O notation of modifying an existing element in a heap data structure?
Wikipedia says "increase key" is O(log n) for a binary heap, but it is referring to an internal function. I'm not asking about that: Imagine the data structure had an external/public ...
- 6,864
1
vote
1
answer
581
views
Unordered Map with indexing and range based loops [closed]
I am creating a C++ library (more of a passion project to be honest). Its aim is to enhance the existing unordered_map STL by giving it indexing abilities and range-based loops, based on the order of ...
1
vote
1
answer
224
views
High Throughput Concurrent Map Access and Periodic Updates Causing Contention and Latency Spikes
I am working on a Go application where two concurrent maps, products and productCatalog, are accessed by numerous threads in live traffic to retrieve data at high throughput. These maps are populated ...
- 121
10
votes
6
answers
6k
views
How to ensure data consistency in system with multiple databases?
Let's say, in a rather big application suite with multiple more or less integrated products, data is stored across multiple databases. Some of them are SQL-ish DB clusters, some are MongoDB clusters.
...
- 255
0
votes
1
answer
646
views
C++ creating a class with self referencing custom comparator [closed]
This is the node for a tree
class Node {
public:
string name;
map<string, string> attribute;
Node* parent;
set<Node*, customComparison> children;
};
Where customComparison in ...
- 111
-1
votes
1
answer
634
views
What is the best way to store a recursive data structure? [closed]
What is the best way to store a recursive data structure in a program? (in C++)
I'm reading data from file, and want to keep it in memory while I validate the data and the user can manipulate it. It ...
- 107
0
votes
1
answer
194
views
Why is `WeakSet` great for tracking circular references?
In MDN article on WeakSets, an example is given for using weak sets to avoid infinite recursion on circular references:
// Execute a callback on everything stored inside an object
function ...
- 145
6
votes
2
answers
1k
views
Object immutability and persistence
While I was learning functional programming I have encounterd two for me very similar terms: immutability and persistence. Also I have read simular questions from stackoverflow, but I am still ...
- 235
0
votes
2
answers
131
views
How do I control many datasets that are related and can change independently?
Background: I'm a scientist working with a small team on data analytics. No one on our team has any experience with software engineering, though I'm trying to change that.
Suppose I have an array A ...
user400809
1
vote
1
answer
230
views
Trie data for searches in time ranges
I am developing an API responsible for returning the distinct searches made on a service given a date prefix.
Example data:
2015-08-01 00:00:18 %22http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%...
- 111
0
votes
1
answer
782
views
How is it possible to store the AST nodes location in the source code?
I created a simple parser in Rust and defined the AST like this:
enum Expression {
Number(i32),
BinaryOperator(Box<Expression>, Operator, Box<Expression>),
Identifier(String),
}...
- 111
-2
votes
1
answer
76
views
Assigning Relationships between various objects in a collection
Similarly to a sports team chemistry or even a better example would be fifa ultimate team chemistry where putting certain players in the team would improve the team's overall chemistry
Say there are ...
0
votes
1
answer
290
views
Are self-written queues and linked lists really worth it (better than built-in arrays)?
Let's say I need a queue for my project and the language I use does not have a built-in queue structure. So I decided to implement one myself.
So I create an object with elements positions as the ...
- 39
0
votes
2
answers
816
views
How to create an interface in C that can work on two identical structs with differently named fields
Question Background
Consider a scenario in which I have two structs. Both consist of three fields for doubles. The only difference is the names used to refer to these fields. The first struct is for ...
- 111
0
votes
1
answer
137
views
How is model reconstructed with event sourcing in CQRS principle?
I'm trying to understand the CQRS design.
As far as I know now, every time a change to a model is saved, the (state) change is stored as a separate record.
When reading a model, the model is ...
- 331
-3
votes
1
answer
1k
views
What's the right data structure to combine iteration of key-value pairs, and iteration of values
I'm looking for a pre-existing or easy-to-build data structure available in Java, which can do these 2 things efficiently:
Fetch all the values stored in the collection quickly.
Iterate over the data ...
- 167
14
votes
2
answers
4k
views
Is there a common term for a fixed-length, fifo, "push through" array or list?
Here's a ubiquitous data structure in, say, game and UX programming:
there're a fixed number of items (say, "5")
they're all "zero" to begin with
you can put one in on the "...
- 447
-1
votes
1
answer
55
views
How to model multiple requirement scenario
I want to store and analyze university course requirements. Lets say, students to register for ABC123 course following requirements should satisfy.
Must pass CBA321
Should be taken with XYZ123
Must ...
3
votes
4
answers
2k
views
Iterating through stacks and queues?
Are stacks and queues allowed to be iterated over, conceptually?
It “feels wrong” for some reason but I can't come up with a reason why it would be bad to see through the whole thing that's stacked/...
- 159
2
votes
3
answers
236
views
Eliminating combinations based on user input
I am stuck with this particular problem. To give context to the problem, I am developing a mobile app which helps with loading dangerous goods onto a truck. Ignore the size and weight of the dangerous ...
- 29
3
votes
3
answers
524
views
What are the existing theories or methodologies around synchronizing tree structures?
I need to compare and synchronize two tree data structures.
I have Tree A (the "Reference Tree") and Tree B (the "Target Tree"). I need to bring the Target Tree into compliance ...
- 171
0
votes
1
answer
230
views
How is B-Tree used in DBMS-s if WHERE condition can be anything?
The most of the database management systems use B-tree as a data structure for increasing the performance.
Let's imagine that we have a table users with the following columns: id (int), name (string), ...
- 111