WARFRAME Wiki
Advertisement
WARFRAME Wiki
ArchimedeanYonta
“Very well. Keep your secrets.”
This article/section contains unofficial information, concepts, or terminology derived from or based on community discussion, invention, or knowledge. It may be subjective and contain information or terminology that is not used by Digital Extremes or in official WARFRAME communications, and may not be an officially recognized concept.

This meta article serves to document how to get some item metadata from overframe.gg. For some reason, their website exposes some item data within the HTML source, allowing users to view data that is not available by WARFRAME developers through official channels like Public Export or in-game. Because of this, overframe.gg is a valuable resource in learning some truths in the underlying mechanics and stats of items which is sometimes used to independently validate or contribute new content on the wiki (for example, the infamous pre-Update 35.5 (2024-03-27) Accuracy stat calculation in the arsenal).

Note that the wiki is not affiliated with overframe.gg in any way so use this information at your risk.

History[]

Overframe is a build calculator website announced on April 9, 2019[1] (just after Update 24.6 (2019-04-04) with Plains of Eidolon remaster for context; don't have historical data for any game changes before that version). It is designed as a item/character build tool and an item/character build database where users can submit their builds to share to the internet. Its interface is localized in all the languages that WARFRAME supports.

Informally, it served as a successor to other community-based build tools such as the defunct warframe-builder.com and tennoware.com which were used back in the day.

For the most part, it keeps up-to-date with WARFRAME builds with at most a week or two delay in updating their website with new content. Rarely the website exposes new content that is not accessible in the game yet (a recent example would be showing Incarnon mode stats for some weapons before Update 31.5 (2022-04-27)[2]). This implies that overframe.gg has access to some non-public data on WARFRAME.[3]

The website is hosted by Magic Find in the Mobafire Network since its inception.[4]

As of 03:19, 5 January 2024 (UTC), its current lead developer is Semlar who also known in the WARFRAME community for their Riven price tracker[5][6] among other community-facing tools.

Accessing Metadata[]

This is best done on any desktop browser application with access to browser development tools (e.g. https://developer.mozilla.org/en-US/docs/Learn/Common_questions/Tools_and_setup/What_are_browser_developer_tools). Here are some important links for quick navigation:

Endpoint Structure[]

Each item page has an endpoint with this structure: /items/arsenal/[id]/[[...slug]]. The URL slug can be omitted and users can still access the item's page with the corresponding id. For example:

  • NovaIcon272 Nova has an id of 1 and can be accessed from https://overframe.gg/items/arsenal/1/ or https://overframe.gg/items/arsenal/1/nova/
  • (Latest item added as of Hotfix 35.6.1 (2024-05-16)) KuvaSobek Kuva Sobek has an id of 6554 and can be accessed from https://overframe.gg/items/arsenal/6554/ or https://overframe.gg/items/arsenal/6554/kuva-sobek/

When new items are introduced, these will be assigned IDs in the order in which they are added. In other words, ID assignment is not random. Example of the first 62 IDs which are all the playable avatars prior to Update 23.5 (2018-08-24) (note that there is no RevenantIcon272 Revenant until ID 1378):

View List
1, Nova
2, Nova Prime
3, Elytron
4, Odonata Prime
5, Odonata
6, Itzal
7, Amesha
8, Banshee
9, Banshee Prime
10, Octavia
11, Valkyr
12, Valkyr Prime
13, Atlas
14, Mesa
15, Chroma
16, Ember
17, Ember Prime
18, Excalibur
19, Excalibur Prime
20, Excalibur Umbra
21, Titania
22, Frost
23, Frost Prime
24, Gara
25, Mirage
26, Mirage Prime
27, Nidus
28, Nyx
29, Nyx Prime
30, Khora
31, Loki
32, Loki Prime
33, Mag
34, Mag Prime
35, Limbo Prime
36, Limbo
37, Wukong
38, Nekros
39, Nekros Prime
40, Nezha
41, Ash Prime
42, Ash
43, Oberon
44, Oberon Prime
45, Hydroid Prime
46, Hydroid
47, Harrow
48, Ivara
49, Rhino
50, Rhino Prime
51, Inaros
52, Saryn
53, Saryn Prime
54, Zephyr
55, Zephyr Prime
56, Vauban
57, Vauban Prime
58, Trinity
59, Trinity Prime
60, Volt
61, Volt Prime
62, Equinox

However, there are some oddities:

  • https://overframe.gg/items/arsenal/6549/ gets redirected to https://overframe.gg/items/arsenal/6548/ (VeloxPrime Velox Prime)
  • https://overframe.gg/items/arsenal/5555/ gets redirected to https://overframe.gg/items/arsenal/5554/ (Ambassador Ambassador)
  • Unknown Augments:
  • Unknown Arcanes:
    • https://overframe.gg/items/arsenal/6497/
    • https://overframe.gg/items/arsenal/6498/
    • https://overframe.gg/items/arsenal/6499/
    • https://overframe.gg/items/arsenal/6500/
    • https://overframe.gg/items/arsenal/6501/
    • https://overframe.gg/items/arsenal/6502/
    • https://overframe.gg/items/arsenal/6503/
    • https://overframe.gg/items/arsenal/6504/
    • https://overframe.gg/items/arsenal/6505/
    • https://overframe.gg/items/arsenal/6506/

Item IDs[]

Main article: WARFRAME Wiki:Overframe/Item IDs

HTML Source as JSON[]

Whenever you load the modding interface for an item, there is usually JSON data is cached locally on the client in these tags: <script id="__NEXT_DATA__" type="application/json"></script> (if you don't see the data associated with an item, purge your browser's cache).[7]

You can use your favorite JSON reader to pretty format for human readability.

The data shown may or may not be a one-to-one representation on how the game stores item data internally, but if you cross-check with in-game testing and looking at what's available within the in-game UI, readers can validate that most of the information has truthfulness to them and can be trusted (see WARFRAME Wiki:Research on some guidance). Key names do not match in-game UI but readers can infer using contextual clues and knowledge on game mechanics (for example, you may see that a lot of hit-scan weapons have "traceDistance": 300 which we can derive "traceDistance" as maximum range of a weapon in meters through in-game testing or knowing about the patch history of the game[8]).

Example[]

For example, the following code block contains the relevant Braton Braton's data from https://overframe.gg/build/new/1229/braton/ as of 03:19, 5 January 2024 (UTC) (there are additional data outside of the item object that contains metadata on the webpage/web app itself). Some of these fields are documented on the respective item database/data store on the wiki (see Template:ModuleNav for navigation box to these pages).

            "item": {
                "categories": ["weapon", "primary", "primary-rifle"],
                "data": {
                    "AmmoClipSize": 45,
                    "ArtifactSlots": ["AP_UNIVERSAL", "AP_UNIVERSAL", "AP_UNIVERSAL", "AP_UNIVERSAL", "AP_UNIVERSAL", "AP_UNIVERSAL", "AP_UNIVERSAL", "AP_UNIVERSAL", "AP_TACTIC", "AP_UNIVERSAL"],
                    "AvailableOnPvp": 1,
                    "Behaviors": [{
                            "fire:Type": "/EE/Types/Game/WeaponTraceFireBehavior",
                            "fire:WeaponTraceFireBehavior": {
                                "AIMED_ACCURACY": {
                                    "Spread": {
                                        "SHOOTING": {
                                            "range": [2, 5],
                                            "type": "ST_EXPONENTIAL"
                                        }
                                    }
                                },
                                "IgnoreFireIterations": 0,
                                "IsMeleeBehavior": 0,
                                "IsSilenced": 0,
                                "RoundUpAmmoConsumption": 0,
                                "ScaleAmmoRequirement": 0,
                                "UseAmmo": 1,
                                "ammoRequirement": 1,
                                "ammoType": "/Lotus/Weapons/Ammo/RifleAmmoEx",
                                "fireIterations": 1,
                                "traceDistance": 300,
                                "tracePunctureDepth": 0
                            },
                            "impact:LotusWeaponImpactBehavior": {
                                "AttackData": {
                                    "Amount": 24,
                                    "DT_IMPACT": 0.33,
                                    "DT_PUNCTURE": 0.33,
                                    "DT_SLASH": 0.34,
                                    "HitType": "DHT_NONE",
                                    "ProcChance": 0.06,
                                    "Type": "DT_PHYSICAL",
                                    "UseNewFormat": 0
                                },
                                "PlayerDamageMultiplier": 1,
                                "PvpDamageMultiplier": 1.38,
                                "criticalHitChance": 0.12,
                                "criticalHitDamageMultiplier": 1.6,
                                "radius": 0
                            },
                            "impact:Type": "/Lotus/Types/Weapon/LotusWeaponImpactBehavior",
                            "state:Type": "/EE/Types/Game/WeaponAutomaticStateBehavior",
                            "state:WeaponAutomaticStateBehavior": {
                                "BehaviorTag": "/Lotus/Language/Menu/Loadout_TriggerAuto",
                                "IsAlternateFire": 0,
                                "LocTag": "/Lotus/Language/Menu/Loadout_TriggerAuto",
                                "fireRate": 525,
                                "reloadTime": 2
                            }
                        }, {
                            "fire:RadialOnMissTraceFireBehavior": {
                                "AIMED_ACCURACY": {
                                    "Spread": {
                                        "SHOOTING": {
                                            "range": [0.3, 0.7],
                                            "type": "ST_EXPONENTIAL"
                                        }
                                    }
                                },
                                "CentralBeam": 0,
                                "IgnoreFireIterations": 0,
                                "IsMeleeBehavior": 0,
                                "IsSilenced": 0,
                                "MultiBeam": 1,
                                "RoundUpAmmoConsumption": 0,
                                "ScaleAmmoRequirement": 0,
                                "UseAmmo": 0,
                                "ammoRequirement": 1,
                                "ammoType": "/EE/Types/Game/AmmoEx",
                                "fireIterations": 1,
                                "traceDistance": 300,
                                "tracePunctureDepth": 0
                            },
                            "fire:Type": "/Lotus/Types/Weapon/RadialOnMissTraceFireBehavior",
                            "impact:RadialOnMissImpactBehavior": {
                                "AttackData": {
                                    "Amount": 50,
                                    "DT_IMPACT": 0.4,
                                    "DT_PUNCTURE": 0.04,
                                    "DT_SLASH": 0.56,
                                    "HitType": "DHT_NONE",
                                    "ProcChance": 0.12,
                                    "Type": "DT_PHYSICAL",
                                    "UseNewFormat": 0
                                },
                                "PlayerDamageMultiplier": 1,
                                "PvpDamageMultiplier": 1,
                                "RadialDamage": {
                                    "DamagePercent": {
                                        "DT_CINEMATIC": 0,
                                        "DT_CORROSIVE": 0,
                                        "DT_ELECTRICITY": 0,
                                        "DT_ENERGY_DRAIN": 0,
                                        "DT_EXPLOSION": 0,
                                        "DT_FINISHER": 0,
                                        "DT_FIRE": 1,
                                        "DT_FREEZE": 0,
                                        "DT_GAS": 0,
                                        "DT_HEALTH_DRAIN": 0,
                                        "DT_IMPACT": 0,
                                        "DT_MAGNETIC": 0,
                                        "DT_POISON": 0,
                                        "DT_PUNCTURE": 0,
                                        "DT_RADIANT": 0,
                                        "DT_RADIATION": 0,
                                        "DT_SENTIENT": 0,
                                        "DT_SHIELD_DRAIN": 0,
                                        "DT_SLASH": 0,
                                        "DT_VIRAL": 0
                                    },
                                    "baseAmount": 50,
                                    "baseProcChance": 0.12,
                                    "checkForCover": 1,
                                    "criticalChance": 0.2,
                                    "criticalMultiplier": 2,
                                    "fallOff": 0.1,
                                    "ignoreSource": 1,
                                    "radius": 3
                                },
                                "criticalHitChance": 0.3,
                                "criticalHitDamageMultiplier": 3,
                                "radius": 0
                            },
                            "impact:Type": "/Lotus/Types/Weapon/RadialOnMissImpactBehavior",
                            "state:Type": "/EE/Types/Game/WeaponAutomaticStateBehavior",
                            "state:WeaponAutomaticStateBehavior": {
                                "BehaviorTag": "/Lotus/Language/Mods/ZarimanRifleAltFireUnlockEvo",
                                "IsAlternateFire": 0,
                                "LocTag": "/Lotus/Language/Menu/Loadout_TriggerAuto",
                                "fireRate": 300,
                                "reloadTime": 2
                            }
                        }
                    ],
                    "CompatibilityTags": ["ASSAULT_AMMO"],
                    "EquipTime": 1.1,
                    "FireModes": [{
                            "behaviorIndex": 0,
                            "localizedTag": "/Lotus/Language/Menu/Loadout_TriggerAuto"
                        }, {
                            "behaviorIndex": 1,
                            "localizedTag": "/Lotus/Language/Mods/ZarimanRifleAltFireUnlockEvo"
                        }
                    ],
                    "Icon": "/Lotus/Interface/Icons/StoreIcons/Weapons/PrimaryWeapons/Weapons/Braton.png",
                    "InventorySlot": "SLOT_2",
                    "LocalizeDescTag": "/Lotus/Language/Items/RifleDesc",
                    "LocalizeTag": "/Lotus/Language/Items/RifleName",
                    "MarketMode": "MM_VISIBLE",
                    "OmegaAttenuation": 1.35,
                    "PVPAmmoClipSize": 30,
                    "ProductCategory": "LongGuns",
                    "RegularPrice": 25000,
                    "SellingPrice": 7500
                },
                "id": 1229,
                "parent": "/Lotus/Weapons/Tenno/Rifle/LotusAssaultStandardRifle",
                "parents": ["/Lotus/Weapons/Tenno/Rifle/LotusAssaultStandardRifle", "/Lotus/Weapons/Tenno/Rifle/LotusAssaultRifle", "/Lotus/Weapons/Tenno/Rifle/LotusRifle", "/Lotus/Weapons/Tenno/LotusLongGun", "/Lotus/Weapons/Tenno/LotusBulletWeapon"],
                "path": "/Lotus/Weapons/Tenno/Rifle/Rifle",
                "storeData": {
                    "AvailableOnPvp": 1,
                    "Giftable": 0,
                    "ProductCategory": "LongGuns",
                    "RegularPrice": 25000,
                    "SearchTags": ["/Lotus/Language/Items/AssaultRifleCategoryName", "/Lotus/Language/Items/RifleCategoryName"],
                    "SellingPrice": 7500
                },
                "storeItemType": "/Lotus/StoreItems/Weapons/Tenno/Rifle/Rifle",
                "tag": "Weapon",
                "texture": "/Lotus/Interface/Icons/Store/CorpusAutoRifle.png",
                "texture_new": "/Lotus/Interface/Icons/StoreIcons/Weapons/PrimaryWeapons/Weapons/Braton.png"
            }

Client-side JavaScript[]

If you look inside the JavaScript debugger and navigate to static.overframe.gg/_next/static/chunks/db, you will find some JS scripts that parse a JSON string (acting as some sort of database or data store).

  • abilities.<hash>.js - Abilities
  • items.<hash>.js - all items
  • moddescriptions.<hash>.js - mod descriptions
  • mods.<hash>.js - Mods
  • modsets.<hash>.js - Set Modss
  • modularparts.<hash>.js - Modular Weapons
  • patchlogs.<hash>.js - patch history
  • rivens.<hash>.js - Riven Mods

Notable Discoveries[]

  • Some understanding of how Accuracy stat is affected by mods. There is an internal Spread stat that Accuracy modifiers act on. In other words, Accuracy is a derived stat.
    • As of Update 35.5 (2024-03-27), this information is exposed in the in-game UI, albeit under hover-over tooltips, as "Deviation With Aim" and "Max Deviation".
  • Hidden Compatibility Tags which explain why some Warframes, weapons, or companions cannot equip a specific mod even though the mod card says it is for a certain class of items.
  • How specific weapon attacks are stored by the game internally in the form of structured data.
  • Upgrade types.
  • Hidden Damage types and Status Effects.
  • Specific stat operations like ADD and STACKING_MULTIPLY that correspond to what players refer to as "additive stacking" and multiplicative stacking" (Calculating Bonuses).
  • May help explain some weapons' discrepancy in damage calculations when factoring Condition Overload (Mechanic).
  • Specific numbers for unique reload mechanics such as Battery Weapons (e.g. recharge delay, recharge rate) and by-shell reloads (e.g. reload start delay, reload loop time, and reload end delay).
  • Starting Energy as a hidden stat for Warframes/PowerSuits.
    • As of Update 35.5 (2024-03-27), this information is exposed in the in-game UI, albeit under hover-over tooltips.

Web Scraper[]

Quick and dirty web scraper in Go to get JSON data within script tags, process it, and output results into a CSV file.

  • Takes ~30 minutes to go through 6,555 item pages (1918514 ms = 31.97 minutes)
    • Can be optimized by using Goroutines, spawning multiple worker processes, directly unmarshal the entire response body, avoid using RegEx, use a buffered writer to CSV, skip redirects (duplicative information)
    • Sample code in Python and Rust for benchmarking how many times can we download a website in a second (see bench_download_webpage benchmark): https://github.com/thundergolfer/uni/tree/main/performance/computers-are-fast
      • Author concluded 3-4 in a second using Python or Rust, which seems about right comparing against our single-threaded Go program. 6,555 webpages / 3 webpages/sec = 2185 seconds or ~36.4 minutes
  • Assisted using Google Gemini

Source Code[]

// Scraping ids and item names from https://overframe.gg
package main

import (
	"encoding/json"
	"fmt"
	"io/ioutil"
	"log"
	"net/http"
	"os"
	"regexp"
	"time"
)

type ItemData struct {
	Props struct {
		Item struct {
			Name string `json:"name"`
		} `json:"item"`
	} `json:"props"`
}

const START_ID = 1
const MAX_PAGE_ID = 6554
const BASE_URL = "https://overframe.gg/items/arsenal/%d"	// "https://overframe.gg/build/new/%d"

func main() {
	startTime := time.Now() // Start time measurement

	// Open the CSV file for writing
	file, err := os.Create("items.csv")
	if err != nil {
		log.Fatal(err)
	}
	defer file.Close()

	// Write headers to the CSV file
	_, err = file.WriteString("id,name\n")
	if err != nil {
		log.Fatal(err)
	}

	// Define a regular expression to find the script tag (inefficient, but gets the job done)
	scriptRegex := regexp.MustCompile(`<script id="__NEXT_DATA__" type="application/json">(.*?)</script>`)
	log.Println(scriptRegex)

	for id := START_ID; id <= MAX_PAGE_ID; id++ {
		url := fmt.Sprintf(BASE_URL, id)

		// Make the HTTP request
		resp, err := http.Get(url)
		if err != nil {
			log.Printf("Error fetching page %d: %v", id, err)
			continue
		}
		defer resp.Body.Close()

		// Read the response body
		body, err := ioutil.ReadAll(resp.Body)
		if err != nil {
			log.Printf("Error reading body for page %d: %v", id, err)
			continue
		}
		
		// Find matches in the body
		matches := scriptRegex.FindSubmatch(body)
		if matches == nil {
			fmt.Println("Script tag with id=\"__NEXT_DATA__\" not found")
			continue
		}
		
		// Extract the captured group (script content)
		scriptContent := string(matches[1])
		
		if scriptContent == "" {
			log.Printf("Script content empty for page %d, skipping", id)
			continue
		}

		// Unmarshal JSON from script content (assuming JSON is within the script)
		var jsonData map[string]interface{}
		err = json.Unmarshal([]byte(scriptContent), &jsonData)
		// log.Println(jsonData)
		if err != nil {
			log.Printf("Error unmarshalling JSON data for page %d: %v", id, err)
			continue
		}

		// Extract item name from JSON data (modify this based on actual data structure)
        // Using type assertions to check type of data stored under each key
		pagePropsObj, ok := jsonData["props"].(map[string]interface{})["pageProps"].(map[string]interface{})
		if !ok {
			log.Printf("props.pageProps not found in JSON data for page %d", id)
			continue
		}

		var itemName string = "nil"
		if itemObj, ok := pagePropsObj["item"].(map[string]interface{}); ok {
		  if name, ok := itemObj["name"].(string); ok {
			itemName = name
		  }
		}

		// Write data to CSV
		csvLine := fmt.Sprintf("%d,%s\n", id, itemName)
		_, err = file.WriteString(csvLine)
		if err != nil {
			log.Printf("Error writing to CSV for page %d: %v", id, err)
		}
	}
    
	fmt.Println("Scraping completed. Results saved to items.csv")
	elapsedTime := time.Since(startTime).Milliseconds() // Measure elapsed time
	fmt.Println("Elapsed time", elapsedTime)
}

Trivia[]

  • https://overframe.gg/build/new/{item_id}/ endpoint accepts any item id, even non-moddable items. For example, https://overframe.gg/build/new/6582/ takes the user to an empty build menu for the Orokin Ballistics Matrix resource. Compatible mod data seems to be produced on demand and not pre-built server-side.
  • It was featured as "best community-made resources" on an official warframe.com blog post.[9]

Solution Stack[]

"The site is written in React (CRA) + TypeScript, with Caddy proxying a Django-Rest-Framework API server on the same port. Hosted as a docker image on AWS Fargate, with code hosted and continuously deployed by Gitlab and Gitlab CI."[10]

References[]

  1. Adys (2019, April 9). Introducing Overframe: A build calculator and a home to share Warframe builds. Reddit. Accessed 2024-01-05. Archived from the original on 2024-01-05.
  2. (2023, January 3). NEW BUILD: BRATON. Overframe. Archived from the original on 2024-01-05. Archived snapshot of the Braton build page before U31.5 release. You can see in HTML source that it includes Incarnon mode stats. Note that these stats are not same post-release.
  3. (2024, July 23). Akmagnus Prime. Overframe. Accessed 2024-07-25. Archived from the original on 2024-07-25. DE accidentally leaked stats for Akmagnus Prime and Semlar said in the Overframe Discord that "DE leaked the Akmagnus Prime in their own patch notes so I'm going to consider this fair game for the website".
  4. (2024, May 25). overframe.gg. Whois.com. Accessed 2024-05-25. Archived from the original on 2024-05-25. WHOIS lookup says Magic Find is the registrant of the overframe.gg domain, registered on 12th October 2018.
  5. Semlar (2019, March 4). Website Status Update 1.0. Patreon. Accessed 2024-01-04. Archived from the original on 2024-01-05.
  6. Ford, Rebecca (2019, March 6). Riven Trading & Toolbuilders: Coming Changes. Warframe Forums. Accessed 2024-01-05. Archived from the original on 2024-01-05.
  7. jensmeindertsma (2020, July 13). Why does __NEXT_DATA__ exists?. GitHub. Accessed 2024-05-25. Archived from the original on 2024-05-25. "That is used so Next (well, React) knows what data you want to hydrate on your page during server side rendering. Otherwise the server would not know what data to use to populate your React components. The page needs to get the data from somewhere and the DOM makes the most sense for any server rendered JavaScript application." - jamesmosier.
  8. "All trace fire weapons trace distance reduced to 300m apart from Sniper Rifles which stay at 1000m." - Update 22.0 (2017-10-12) patch notes
  9. (2022, February 7). Warframe Tools and Resources Every Player Should Know About. Digital Extremes. Accessed 2025-01-05. Archived from the original on 2025-01-05.
  10. Leclanche, Jerome (2019, April 9). Introducing Overframe: A build calculator and a home to share Warframe builds. Reddit. Accessed 2024-01-05. Archived from the original on 2024-02-24.
Advertisement