How 'Deque' Saved My Python Script from Slowing to a Crawl
programming·@makerhacks·
0.000 HBDHow 'Deque' Saved My Python Script from Slowing to a Crawl
 Sometimes the right tool makes all the difference. I was building a Python script that hit API rate limits a bit too often. Not because I was doing anything wild, but because checking and managing timestamps of previous API calls started to slow things down. The culprit? A regular Python list. The fix? A simple but powerful data structure called `deque` (pronounced *“deck”*). If you’re dealing with queues, rate limits, or just need to add/remove items from both ends of a list quickly, this little built-in feature might be exactly what you need. ## What is `deque`? `deque` stands for **Double-Ended Queue**. It lives in Python’s `collections` module, and you use it like this: ```python from collections import deque ``` It works a lot like a list—but it’s built for speed, especially when you’re adding or removing items from **either end**. ## How is it different from a regular list? Here’s the short version: | Operation | List Time Complexity | `deque` Time Complexity | |------------------------------|----------------------|--------------------------| | `append()` (end) | Fast (O(1)) | Fast (O(1)) | | `pop()` (end) | Fast (O(1)) | Fast (O(1)) | | `insert(0, item)` (start) | **Slow** (O(n)) | `appendleft()` = Fast (O(1)) | | `pop(0)` (start) | **Slow** (O(n)) | `popleft()` = Fast (O(1)) | The difference is massive when you’re working with larger data, especially when you’re removing items from the front. Lists have to shift every other element to close the gap. `deque` just updates a pointer. ## Under the hood: Why is `deque` faster? Python lists are dynamic arrays. They’re great at random access (getting item by index), but not so great at adding/removing items at the front. That’s because arrays need to keep elements in order in memory, so any time you insert at the front, everything else has to slide down one spot. `deque`, on the other hand, is more like a chain of blocks (technically a doubly-linked list under the hood). Adding or removing from either end is a matter of updating pointers—no shifting required. ## Rate Limiting with `deque` Here’s what I was dealing with: I needed to track the **timestamps of recent API calls**—say, only the ones made in the last 60 seconds. That means: - Every time I make a call, I need to **add** the current time. - Before making a call, I need to **remove** any timestamps older than 60 seconds. This is a classic sliding window problem. ### Why `deque` works perfectly here: ```python from collections import deque from time import time GEMINI_CALL_TIMESTAMPS = deque() def can_make_call(): now = time() # Remove old timestamps while GEMINI_CALL_TIMESTAMPS and now - GEMINI_CALL_TIMESTAMPS[0] > 60: GEMINI_CALL_TIMESTAMPS.popleft() # O(1) # Check if under limit if len(GEMINI_CALL_TIMESTAMPS) < 10: GEMINI_CALL_TIMESTAMPS.append(now) return True else: return False ``` If I’d used a list here and called `pop(0)` instead of `popleft()`, the cleanup step would get slower the more timestamps there were—because lists shift everything after the first item. That’s a performance hit you *feel* when you're doing this frequently. ## size limits with `maxlen` You can also give your `deque` a fixed size like this: ```python history = deque(maxlen=100) ``` Now, when it’s full and you add a new item, it automatically drops one from the other end. Great for logs, recent history, or metrics. ## TL;DR If you need fast operations at **both ends** of a sequence, `deque` is your friend. It’s especially handy when: - You’re implementing a **queue** or **stack**. - You want a **sliding window** of data. - You need to regularly remove the oldest items from a growing collection. In my case, switching to `deque` made my rate limiter clean, fast, and reliable—no more slowdowns from shifting long lists. It’s a small change, but it made a big difference.
👍 khalstem, slider2990, smartvote, pewsplosions, brofund, brofi, upfundme, technologix, bozz.sports, stemisaria, nfttunz, denizcakmak, iktisat, baltai, travelpic, surrealis, lordnasty, beardoin, dab-vote, brofund-witness, dailydab, clubvote, jacobtothe, antisocialist, freebornsociety, yayogerardo, zakludick, kenny-crane, steevc, marsupia, leighscotford, slobberchops, joeyarnoldvn, usainvote, darth-azrael, darth-cryptic, retrodroid, mytechtrail, edicted, iansart, anonsteve, hextech, enforcer48, admiralbot, steemflagrewards, memehub, aninsidejob, steemseph, nftspecialists, calatorulmiop, lemouth, steemstem-trig, steemstem, dna-replication, roelandp, stemsocial, minnowbooster, howo, aboutcoolscience, robotics101, alexdory, melvin7, deholt, temitayo-pelumi, stem.witness, metabs, curie, techslut, walterjay, valth, dhimmel, mobbs, sustainablyyours, helo, alexander.alexis, tsoldovieri, abigail-dantes, aidefr, sorin.cristescu, splash-of-angs63, kenadis, r00sj3, sco, intrepidphotos, emiliomoron, geopolis, francostem, gadrian, de-stem, pboulet, crowdwitness, pinkfloyd878, inibless, samminator, madridbg, nattybongo, profwhitetower, noelyss, boxcarblue, techken, enzor, adelepazani, croctopus, superlotto, qberry, robmolecule, doctor-cog-diss, cloh76, tfeldman, bitrocker2020, coindevil, meno, utube, dcrops, mugueto2022, marc-allaria, zipporah, kgakakillerg, rem-steem, beta500, tawadak24, aries90, jhymi, rhemagames, kevinwong, justyy, michelle.gent, princessmewmew, investingpennies, the.success.club, monica-ene, altleft, rt395, neumannsalva, putu300, nateaguila, musicvoter2, armandosodano, steemstorage, yozen, jayna, empath, bartosz546, gerdtrudroepke, kristall97, double-negative, carlos13, blingit, justfavour, kryptof, modernzorker, pipiczech, felt.buzz, fineartnow, takowi, greddyforce, talentclub, meritocracy, drricksanchez, merit.ahama, cryptofiloz, the100, quinnertronics, lukasbachofner, azj26, stayoutoftherz, rocky1, hetty-rowan, dynamicrypto, therising, cryptocoinkb, hairgistix, okluvmee, humbe, dandesign86, zyx066, steemvault, bscrypto, multifacetas, tokensink, omarrojas, vickoly, callmesmile, clpacksperiment, someguy123, yadamaniart, ohamdache, arunava, gunthertopp, sunsea, pandasquad, bflanagin, kylealex, instagram-models, seinkalar, yixn, ambicrypto, eliel, photohunt, bil.prag, steemcryptosicko, cakemonster, blackdaisyft, metroair, softa, littlesorceress, failingforwards, newilluminati, belug, steveconnor, cheese4ead, steemean, dawnoner, minerthreat, cliffagreen, michelmake, driptorchpress, migka, detlev, enjar, gabrielatravels, neneandy, movingman, acgalarza, jijisaurart, maickto, chipdip, communitybank, sportscontest, fantasycrypto, juancar347, oscarina, qwerrie, hive-199963, lichtkunstfoto, kiemurainen, goblinknackers, xeldal, rmach, mcsvi, soyjoselopez, adol, enki, killerbot, ibt-survival, mproxima, sanderjansenart, primersion, reggaesteem, holovision.cash, borniet, carilinger, pladozero, jerrybanfield, smariam, juecoree, stem-espanol, iamphysical, aleestra, lorenzor, azulear, ydavgonzalez, miguelangel2801, tomastonyperez, elvigia, josedelacruz, erickyoussif, hmayak, andrick, acont, fran.frey, giulyfarci52, aqua.nano, uche-nna, wasined, psicoluigi, buttcoins, aiziqi, gribouille, vaultec, eric-boucher, robertbira, imcore, eternalsuccess, ennyta, endopediatria, bluefinstudios, m1alsan, gamersclassified, hannes-stoffel, lolz.byte, magic.byte,