The layers at a glance
The request pipeline
Every method call onTikTokClient passes through the same steps.
1
Cache
The client checks
CacheManager first. A recent result for the same input is returned immediately.2
Queue
The call joins
RequestQueue, which limits how many requests run at once (concurrency) and runs higher-priority tasks first.3
Rate limiter
RateLimiter is a token bucket. It allows a set number of requests per second, with short bursts up to twice that rate.4
Circuit breaker
CircuitBreaker watches for repeated failures. After several failures in a row it “opens” and fails fast for a cooldown period instead of hammering TikTok. It then lets a trial request through and closes again once requests succeed.5
Proxy and retries
The client picks a proxy from
Proxymanager, runs the scraper, and reports how the proxy performed. If the attempt fails, it retries up to maxRetries times, usually with a different proxy.6
Format
The raw TikTok item is turned into a clean object such as
TikTokVideo, with formatted dates, durations, available video formats, subtitles, and image posts.Two ways to get data
Each scraper tries the strongest method available and falls back to the next one.- With a Playwright session
- Without a session
If you pass a ready Playwright session, the scraper first calls TikTok’s own web API (for example
/api/item/detail/) from inside the browser page. This is the most reliable path because the request carries real browser cookies and a valid signature.If that call fails, the session can render the page in the browser and hand the HTML to the ParserEngine.ParserEngine: finding the data in the page
TikTok embeds the data for each page as JSON inside a<script> tag, but the exact location changes over time and even between regions. The ParserEngine tries several strategies in order and stops at the first one that works:
- Universal data: reads the
__UNIVERSAL_DATA_FOR_REHYDRATION__script. It checks the known paths first, then searches the whole object for anything that looks like a video item. - SIGI state: reads the older
SIGI_STATEscript. - Inline scripts: scans the page’s other scripts for item data.
- LD+JSON: falls back to the structured-data block TikTok includes for search engines.
TikTokParseError that says how many strategies were tried.
Signing requests: X-Bogus
TikTok’s web API rejects requests without a validX-Bogus signature. The SDK handles this in two ways:
- Inside a Playwright session, it waits for TikTok’s own signing function (
byted_acrawler.frontierSign) to load in the page and asks it to sign each URL. It also uses themsTokencookie TikTok sets in the browser. - Without a browser, the
Signermodule computes theX-Bogusvalue locally from the query string and user agent.signandsignUrlare exported if you need them directly.
Proxies
By default, the SDK fetches a free proxy list from the Proxifly project and refreshes it every few minutes.Proxymanager rotates through the list using round-robin, random, or lowest-failures, and tracks which proxies succeed, how fast they are, and which ones fail. You can pass your own proxies instead, which is recommended for production. See Proxies.
Playwright sessions in detail
When you start aPlaywrightSession, it:
- launches Chromium with automation flags turned off,
- injects a stealth script that sets realistic
navigatorvalues (platform, languages, memory, CPU cores), - opens TikTok’s For You page to collect the
msTokenand CSRF cookies, - builds the standard TikTok web parameters (app ID, device ID, region, screen size, and so on),
- and recreates the page automatically if it crashes.
Next: Installation
Install the SDK and make your first request.
