WARFRAME Wiki:Updating Databases

“Hunter, I have temporarily disabled that ability.”

This article contains JavaScript scripts that users can run locally in their browser's console or machine. As a warning, which goes for any scripts you copy/run from the Internet, MAKE SURE YOU UNDERSTAND THE CODE BEFORE RUNNING IT LOCALLY FOR YOUR OWN SECURITY! Contact an admin if you have any concerns or questions about a script.

This wiki contains many databases and structured data regarding WARFRAME's contents and requires monthly updates to reflect on the accurate data of the game's contents. This article is meant to help editors understand how these databases interact with content on the rest of the wiki as well as how to update them. No programming experience nor wiki editing experience is needed to update our databases.

Contents of /data subpages can be downloaded to your local machine by adding the ?action=raw query string to the end of the URL of the respective subpage. These can be converted to other formats like JSON or CSV using external Lua libraries such as json-lua or cerial.lua. As a disclaimer, use the aforementioned libraries at your own risk as the wiki is not responsible for any damages that may occur.

Structured data stored on the wiki is not meant to reflect 100% on internal implementation or data of the game. Changes are made at editors' discretion to produce meaningful information that can be relayed to readers in accessible formats on the wiki and the wiki only. Data provided by DE through official means like Public Export will not necessarily contain the same information as the wiki and vice versa.

Last updated: Sun, 14 Jan 2024 02:10:26 +0000 (UTC) by User:Cephalon Scientia

List of Databases

Databases are denoted by /data which is the database subpage of a particular Lua module page.

How To Update

Generally, you do not have to be logged-in to edit /data subpages on the wiki. However, some highly dependent data require editors to be logged-in as an autoconfirmed user on the wiki to edit. This is to prevent vandalism and to uphold the integrity of our data. To be an autoconfirmed user, Fandom accounts must be at least four days old and had made 10 edits around the wiki before being able to edit these database pages. Otherwise, the contents of these pages will be publicly available in read-only mode.

To quickly find the right table entry to update, use your browser's CTRL + F hotkey or the wikitext source editor's search button in the toolbar under the "Advanced" tab.

If you make any syntax errors, the editor will alert you after you attempt to submit changes. Just navigate to the line number where it tells you an error occured and add the missing tokens before resubmitting. Common syntax errors include:

• Missing a comma (,) at the end of key-value pairs
• Forgetting a closing curly bracket (}) for tables
• Invalid key name; if you want spaces or special characters in the key, make the key a string and put it inside square brackets like so: ["Key with spaces and colon:"] = "Some value"

Background

Our databases are in the form of Lua tables which have a similar syntax as JSON files, albeit tokens and delimiters are based on the Lua programming language.

Sample table:

TableKey = {
	KeyName = "Some string value",
    ["Another Key Name w/ Spaces"] = true,
    AnotherKeyName = { "Table values", "preferably", "be the same", "data type" },
    Integer = 2,
    Float = 3.1415926535897
}

These tables are parsed by the respective Lua module to be outputted to wikitext on pages that {{#invoke:}} a function from that particular module. For example, Module:Weapons/infobox uses data from Module:Weapons/data to generate the infobox for a particular weapon. Template:WeaponInfoboxAutomatic calls the buildInfobox() function in {{#invoke:Weapons/infobox|buildInfobox}} so any weapon article that uses/transcludes Template:WeaponInfoboxAutomatic will automatically generate the wikitext for the infobox as seen on the Braton article.

Possible Entries

There are limitations to the type of data that can be stored in these databases. Lua tables only support the following data types as values that are mapped to keys or as indexed elements:

• Boolean (e.g. true or false)
• Integer number (e.g. 10)
• Float/decimal number (e.g. 2.5)
• String (e.g. "Text")
• Table (e.g. { "This", "is", "a", "table with", 6, "elements" })
• Nil or null (e.g. nil)

Despite being able to be store functions in Lua tables, MediaWiki does not allow modules to read tables with functions as values so avoid adding these at all costs.

Data Pipeline

The wiki follows a three-tier architecture when handing data stored on the wiki's data stores.

How game data is transferred and processed on the wiki

Source	Storage (Data Layer)	Processing (Business Logic Layer)	Use (Presentation Layer)
WARFRAME client (in-game) Arsenal stats Codex entries Market entries Player research (see WARFRAME Wiki:Research) EE.log (Evolution Engine logs) Official Drop Tables on https://warframe.com/droptables Official Warframe forums on https://forums.warframe.com/ Official Warframe site on https://www.warframe.com/ Mostly for sourcing official image assets, particularly related to Warframes World State endpoint Warframe Community Developers' warframe-worldstate-data intermediary Public Export endpoint Warframe Community Developers' warframe-items intermediary https://kasumata.ee/ - unofficial Warframe sound index https://overframe.gg/ - exposes its own application data which contains data that looks to be sourced from developers Can find JSON on Warframes, weapons, and resources Client-side scripts may use developer names for content like item file paths and internal code names for enums Word-of-mouth from players Direct contact with developers (sometimes community team provide assets)	Data on game objects are manually audited and persisted to on-wiki /data submodules by volunteer wiki editors Media assets uploaded to File namespace via Special:Upload by volunteer wiki editors (or via human-operated bots) These are stored on actual databases/persistent storage + distributed via CDNs; wiki editors will never worry about this infrastructure In the case of front-end scripts, sometimes data is stored within source code in arrays or JSON format	Lua modules and submodules access data via module imports and manipulate them in function calls These are open-sourced and are likely unprotected for volunteer editors/developers to contribute to Client-side scripts may make use of MediaWiki's Action API to fetch data from articles and process them for interactive elements	Templates are transcluded on articles which call a particular module's function to generate wikitext/HTML of data to display to readers Very rarely a module's function is directly called on an article without the use of a template wrapper

Maintaining and Updating

The following are the most important databases to update in order of priority:

1. Module:DropTables/data - WARFRAME Drop Tables
   • Lots of articles about WARFRAME items depend on this for drop chances and locations
     • Void Relic, mission type pages like Survival, planet pages like Earth
     • Template:ArcaneBox, Template:Enemy, Template:ModBox, Template:ResourceInfobox
2. Module:Weapons/data - used by weapon articles, Weapon Comparison, Template:WeaponNav, and tooltips
   • Lots of people rely on the comparison table for Disposition values (Riven Mods), comparing weapon stats, and determining what is "meta"
   • Module:Weapons/ppdata also needs to be updated when Module:Weapons/data is updated (need a manual edit to persist preprocessed data; we don't have a dedicated bot account/webhook for this)
3. Module:Void/data - used by relic articles, Prime weapon/Warframe articles, Void Relic, and tooltips
   • These pages have high traffic whenver a new Prime Access or unvaulting is released since the in-game Codex is not very specific on drop chances of relics
4. Module:Mods/data - used by mod articles and tooltips
5. Module:Warframes/data - used by Warframe articles and tooltips
6. Any other database related to content that was recently updated (e.g. Module:Baro/data if Baro Ki'Teer appears with new items)

Update Frequency

The frequency of updates are about once every three months (i.e. every quarter to match new Prime Access release) or every time a major mainline update is released by the developers. Minor updates can be made during downtime to fix values or to add additional data. Note that it is easier to add more data than to remove existing ones; wiki articles may already depend on old data to generate the pages' content so removing old/depreciated data is a more difficult task and would most likely require additional development time in the respective module pages.

New Content Update Frequency

Database	Brief Description	Approximate Frequency
Module:Version/data	WARFRAME version builds	Daily or weekly depending on hotfix/update schedule
Module:Baro/data	Baro Ki'Teer visits	2 weeks, every other Friday when Baro Ki'Teer visits
Module:Weapons/data/primary	Primary Weapons data	1-3 months, every other mainline update; Disposition update every 3 months with Prime Access
Module:Weapons/data/secondary	Secondary Weapons data	1-3 months, every other mainline update; Disposition update every 3 months with Prime Access
Module:Weapons/data/melee	Melee Weapons data	1-3 months, every other mainline update; Disposition update every 3 months with Prime Access
Module:Weapons/data/archwing	Archgun and Archmelee Weapons data	1-3 months, every other mainline update; Disposition update every 3 months with Prime Access
Module:Weapons/data/companion	Companion Weapons data	Update as needed, when new companion weapons are released
Module:Weapons/data/railjack	Railjack armaments data	Update as needed, when new Railjack weapons are released
Module:Weapons/data/modular	Modular Weapons data	Update as needed, when new modular weapons are released; Disposition update every 3 months with Prime Access
Module:Weapons/data/misc	Other Weapons data that doesn't fit in the other 7 partitions	Update as needed
Module:Weapons/ppdata	Weapons preprocessed data	1-3 months, every other mainline update; Disposition update every 3 months with Prime Access
Module:Blueprints/data	Item blueprints	1-3 months, every other mainline update when a new weapon/item releases
Module:Resources/data	Resources, currencies, foundry components	1-3 months, every other mainline update
Module:DropTables/data	Drop Tables and rewards that are pulled from loot tables	1-3 months, every other mainline update
Module:Enemies/data	Enemies data	1-3 months, every other mainline update
Module:Vendors/data	Syndicate offerings, event shops, and NPC vendors	1-3 months, every other mainline update
Module:TennoGen/data	TennoGen cosmetics	1-6 months, depending on TennoGen schedule
Module:Void/data	Void Relic data	3 months for every Prime Access or every month for Prime Resurgence release
Module:Warframes/data	Warframes data and related playable avatars (Archwings, Necramechs)	3 months, Prime Access release
Module:Ability/data	Character abilities data	3 months, new Warframe release
Module:Cosmetics/data	General Warframe Cosmetics	Update as needed
Module:Codex/data	Codex list	Update as needed
Module:Arcane/data	Arcane Enhancement data	Update as needed, rarely updated
Module:Focus/data	Focus data	Update as needed, rarely updated
Module:Research/data	Clan Research data	Update as needed, rarely updated
Module:Modular/data	Zaws, Kitguns, and other modular items	Update as needed, rarely updated
Module:Conservation/data	Conservation animals	Update as needed, rarely updated
Module:DamageTypes/data	Damage types and Status Effects	Update as needed, rarely updated
Module:Missions/data	Star Chart nodes data	Update as needed, rarely updated
Module:Decorations/data	Decorations data	Update as needed, rarely updated
Module:GuaranteedRewards/data	Rewards that are guaranteed to be rewarded to the player after completing some task (like Daily Tributes or after completing a quest). These are different from drop tables which rely on RNG.	Update as needed, rarely updated

Formatting

• Please use tabs for indentation instead of spaces.
• Read the documentation above each /data subpage for sample formatting of key-value pairs. If there is no documentation written, try your best to follow the convention established in the Lua table entries

Validation

• Some databases have a /data/validate subpage where editors can view to find errors in table entries.

For Developers

Database Design Guidelines

When designing new /data subpages of modules or adding new keys to existing /data tables:

• Key names should be in TitleCase. If a key stores metadata and not data regarding the game's contents (i.e. meant to be used within modules and not presented to readers), prepend an underscore (_) in front of key name.
• Keys should map to one value type and only that value type; do not mix and match value types otherwise it will cause more overhead in checking each individual value's types before performing a certain operation on that value. This also helps with script performance at runtime by reducing the number of conditional checks.
• Tables should not be both array-like and dictionary-like. Stick to one usage of tables for consistent behavior when iterating over values.
• If possible, map values to a key for faster access, especially when tables get extremely big. Internally, Lua implements table types as hash tables.
  • Do not assume that keys represent data nor should they be treated as such. Keys are used to index a specific data table for faster access. It may be possible for an index to share the same value as a key-value pair inside the table, but this does not guarantee anything about the index as data. In most instances, the index will copy the Name key's value since it is more intuitive to look up data entries based on an item's in-game name.
• Keep wikitext/HTML formatting outside of database entries as much as possible. In other words, prohibit (or at the very least limit) the use of wikitext syntax and HTML elements in string values. Databases should contain raw, unprocessed data. Any modifications to this data to display on the wiki should be done in main Lua modules and/or submodules. This is done to decouple data from business logic and user interface design like in a three-tier architecture.
• Exported tables should not contain functions and/or metatables as values. These tables are strictly used to store data in the form of numbers, strings, booleans, and other tables.
  • This restriction is due to mw.loadData(). See mw:Extension:Scribunto/Lua_reference_manual#mw.loadData for a full list of restrictions.
  • However, in the instance where a database is too large to manage, we may allow horizontal partitioning and access table entries through metatables and metamethods. However, databases must be imported via require() instead of mw.loadData(). See Module:Weapons/data for an example of this implementation.
• Since most content in these databases will be presented to wiki readers, avoid storing string values in all capital casing or all lower casing. Use proper sentence casing or title casing instead.
  • Name localization may be an exception since DE stores localized strings in all caps for convention. We should store localized strings in their canonical form as much as possible to prevent loss of information.
• Do not assume that an item's name is the same as its associated article's name on the wiki. Scenarios such as items with the same name, items with similar functions and/or type being represented under one generic article, items with not enough information to justify having its own page, formatting differences, or different localization may cause an article to have a different name than the item.
  • Examples include the mod  Scorch having a page link of Scorch (Mod).
• Tables should allow inversion using an inverted index to support different methods of indexing entries. For example:

  ["Abundant Mutation"] = {
  	BaseDrain = 6,
  	Image = "AbundantMutationMod.png",
  	Introduced = "27.5.4",
  	Link = "Abundant Mutation",
  	MaxRank = 3,
  	Name = "Abundant Mutation",
  	Polarity = "Zenurik",
  	Rarity = "Rare",
  	Tradable = true,
  	Transmutable = false,
  	Type = "Nidus" 
  }
  
  Can be manipulated to be indexed by Type:
  
  ["Nidus"] = {
  	BaseDrain = 6,
  	Image = "AbundantMutationMod.png",
  	Introduced = "27.5.4",
  	Link = "Abundant Mutation",
  	MaxRank = 3,
  	Name = "Abundant Mutation",
  	Polarity = "Zenurik",
  	Rarity = "Rare",
  	Tradable = true,
  	Transmutable = false,
  	Type = "Nidus" 
  }
  
  In the case that a particular inverted key has 
  multiple possible table entries, the inverted key 
  should instead map to a array-like table with each 
  element being a separate table entry.
  

Seeding New Databases

If creating new /data subpages, one could look to writing a script that parses Public Export or https://github.com/WFCD/warframe-items to automate the creation of new Lua tables based on existing JSON data. For Luafying data that exist on one or across many articles on the wiki, one could use Special:Export to export the contents of articles based on category or a manually created list to an XML format for extracting data programmatically. However, if what you are creating is completely new, then unfortunately, you may have to create the Lua tables by hand or write them down in a spreadsheet somewhere before converting it to Lua.

Accessing Data In Lua Modules

We provide two different interfaces to accessing data within Lua modules (and possibly third-party scripts via MediaWiki's Action API, see #Querying and Fetching Data for examples):

1. Using Lua's built-in syntax for table accesses:
   • Index syntax like in most languages

     exampleTable[1] or exampleTable["keyName"]

   • Syntactic sugar with dot notation (doesn't work for numerical indexes)

     exampleTable.keyName

2. Using Module:StatObject as an interface for raw data stored in /data subpages

   local StatObject = require('Module:StatObject')
   
   -- Define getter and format functions in StatObject.default table for 
   -- individual fields present in raw data or computed/derived fields
   StatObject.default = {
   	-- Getter functions:
   	-- Arrays of default values and format strings
   	key = { 'Default Value', '%.2f format' },
       
   	-- Can generate values/formatting with functions
   	-- 'val, ...' is the return from map[1]
   	key = { function(obj) end, function(self, val, ...) end },
       
   	-- If format is a table the return values will be passed to each function/format string
   	-- in the same order they are returned (nil is a pass-through)
   	key = {
   		function(obj) return a, b, c, d end,
   		{ function(self, val) return val end, '%s', nil, '%d', sep = '' }
   	},
       
   	-- Can omit second entry, can omit table
   	key = 'Default Value',
       
   	-- nil means default get (same as omitting)
   	key = nil,
       
       -- Add additional key-value pairs below to define raw getters and format getter functions for data
       -- To use the getters use StatObject.statRead(dataEntry, key) and StatObject.statFormat(dataEntry, key)
   }
   
   local ModData = mw.loadData('Module:Mods/data')		-- Importing sample database
   
   StatObject.statRead(ModData['Serration'], 'Name')	-- Example of getting the raw Name value of Serration mod entry
   StatObject.statFormat(ModData['Serration'], 'Name')	-- Example of getting the formatted Name value of Serration mod entry
   

Querying and Fetching Data

If you want to query these databases, you can write a Lua script to parse and return the data you want in Special:ApiSandbox. MediaWiki's Action API using action=scribunto-console can be used for this purpose.

For example, if one wants to find the names of secondary mods that have the Vazarin (V or ) polarity, the following snippet can be passed into the question parameter:

local ModData = require('Module:Mods/data').Mods
local results = {}
for _, modEntry in pairs(ModData) do
    if (modEntry.Polarity == 'Vazarin' and modEntry.Type == 'Pistol') then
        table.insert(results, modEntry.Name)
    end
end
print(table.concat(results, ', '))

A sample of the above query in ApiSandbox: https://warframe.fandom.com/wiki/Special:ApiSandbox#action=scribunto-console&format=json&title=Module%3AMods%2Fdata&content=&question=local%20ModData%20%3D%20require('Module%3AMods%2Fdata').Mods%0Alocal%20results%20%3D%20%7B%7D%0Afor%20_%2C%20modEntry%20in%20pairs(ModData)%20do%0Aif%20(modEntry.Polarity%20%3D%3D%20'Vazarin'%20and%20modEntry.Type%20%3D%3D%20'Pistol')%20then%0Atable.insert(results%2C%20modEntry.Name)%0Aend%0Aend%0Aprint(table.concat(results%2C%20'%2C%20'))&clear=1

Output to CSV or JSON

The wiki has forked some conversion libraries Module:CSV and Module:JSON for developers' convenience when trying to output the data in CSV or JSON format.

For example, if one wants to get the contents of Module:Mods/data in JSON:

local ModData = require('Module:Mods/data').Mods
local json = require('Module:JSON')
print(json.stringify(ModData))

A sample of the above query in ApiSandbox: https://warframe.fandom.com/wiki/Special:ApiSandbox#action=scribunto-console&format=json&title=Module%3AMods%2Fdata&content=&question=local%20ModData%20%3D%20require('Module%3AMods%2Fdata').Mods%0Alocal%20json%20%3D%20require('Module%3AJSON')%0Aprint(json.stringify(ModData))&clear=1

---

Sample JS using Action API:

let origin = "https://warframe.fandom.com";
let path = "/api.php";
let params = {
	action: "scribunto-console",
	format: "json",
	title: "Module:Mods/data",
	content: "",
	question: `
local ModData = require('Module:Mods/data').Mods
local json = require('Module:JSON')
print(json.stringify(ModData))`,
	clear: 1
};

// [ ["action", "scribunto-console"], ["format","json"], ... ] to "action=scribunto-console&format=json&..."
let queryString = new URLSearchParams([ ...Object.entries(params) ]).toString();

let url = new URL(`${origin}${path}?${queryString}`);

fetch(url)
	.then((data) => data.json())
		.then((json) => {
		if (json.print !== undefined) {	// Replace with json.return if you want to get content from return statement
			console.log(JSON.parse(json.print));
		} else {
			throw json.html;	// Lua script error has occurred
		}
	})
	.catch((error) => console.log(error));

Updating Databases via API

Here are some sample scripts for reference:

Adding New/Missing Keys

For example, if I want to add missing keys in Module:Weapons/data/melee the following script can be run to save the edited database onto your local machine which can be persisted to the database via a manual copy and paste:

// Saves a file to local machine
var saveData = (function() {
	// Creating a temporary DOM element so we can 'click' on an element to download the file.
	// For obvious security reasons, JS running in browser environment does not have direct access to read/writes
	// to local storage. Browser is sandboxed to prevent arbitrary scripts causing damage to clients.
	var a = document.createElement("a");
	// document.body.appendChild(a);
	// a.style = "display: none";
	return function (data, fileName) {
		var json = JSON.stringify(data);
		// Unescaping escape characters
		json = json.replace(/\\n/gm, "\n").replace(/\\t/gm, "\t").replace(/\\"/gm, "\"")
		blob = new Blob([json], {type: "octet/stream"});
		blob = blob.slice(1, blob.size - 1); // Removing first and last quotation mark that designate json as a string
		url = window.URL.createObjectURL(blob);

		a.href = url;
		a.download = fileName;
		a.click();
		window.URL.revokeObjectURL(url);
	};
}() );

let origin = "https://warframe.fandom.com";
let path = "/api.php";
let params = {
	action: "scribunto-console",
	format: "json",
	title: "Module:DropTables/data",
	content: "",
	question: `
local DropData = require('Module:Weapons/data/melee')
local serialize = require('Module:LuaSerializer')
for _, t in pairs(DropData) do
	if (t.Family == nil) then
		t.Family = t.Name
	end
end
print(serialize._serializeTable(DropData))`,
	clear: 1
};

// [ ["action", "scribunto-console"], ["format","json"], ... ] to "action=scribunto-console&format=json&..."
let queryString = new URLSearchParams([ ...Object.entries(params) ]).toString();

let url = new URL(`${origin}${path}?${queryString}`);

fetch(url)
	.then((data) => data.json())
	.then((json) => {
		if (json.print !== undefined) {
			console.log(json.print);
			saveData(json.print, "output.lua");
		} else {
			throw json.html;	// Lua script error has occurred
		}
	})
	.catch((error) => console.log(error));

Updating Table Values

If I want to update Module:DropTables/data entries so that all item drop entries in each particular drop table contain a string of the item type as their second element based on the name of the drop table (e.g. mod drop tables have items of type "Mod"). Note that some items in drop tables do not match the drop table type which is why an explicit value is needed for manual exceptions (e.g. you can technically have resources that drop from the mod drop table; there is loose enforcement on item types in a drop table).

// Saves a file to local machine
var saveData = (function() {
	// Creating a temporary DOM element so we can 'click' on an element to download the file.
	// For obvious security reasons, JS running in browser environment does not have direct access to read/writes
	// to local storage. Browser is sandboxed to prevent arbitrary scripts causing damage to clients.
	var a = document.createElement("a");
	// document.body.appendChild(a);
	// a.style = "display: none";
	return function (data, fileName) {
		var json = JSON.stringify(data);
		// Unescaping escape characters
		json = json.replace(/\\n/gm, "\n").replace(/\\t/gm, "\t").replace(/\\"/gm, "\"")
		blob = new Blob([json], {type: "octet/stream"});
		blob = blob.slice(1, blob.size - 1); // Removing first and last quotation mark that designate json as a string
		url = window.URL.createObjectURL(blob);

		a.href = url;
		a.download = fileName;
		a.click();
		window.URL.revokeObjectURL(url);
	};
}() );

let origin = "https://warframe.fandom.com";
let path = "/api.php";
let params = {
	action: "scribunto-console",
	format: "json",
	title: "Module:DropTables/data",
	content: "",
	question: `
local DropData = require('Module:DropTables/data')
local Serializer = require('Module:LuaSerializer')

for _, collection in ipairs({ 'Containers', 'Enemies' }) do
	for entryName, entry in pairs(DropData[collection] or {}) do
		for _, dropTableName in ipairs({ 'Blueprints', 'Items', 'Mods', 'Relics', 'Resources', 'Sigils'}) do
			for i, dropEntry in ipairs(entry[dropTableName] or {}) do
				-- Remove 's' in dropTableName to get item type
				table.insert(dropEntry, 2, dropTableName:sub(1, -2))
			end
		end 
	end
end

print(Serializer._serializeTable({ Containers = DropData.Containers, Enemies = DropData.Enemies }))`,
	clear: 1
};

// [ ["action", "scribunto-console"], ["format","json"], ... ] to "action=scribunto-console&format=json&..."
let queryString = new URLSearchParams([ ...Object.entries(params) ]).toString();

let url = new URL(`${origin}${path}?${queryString}`);

fetch(url)
	.then((data) => data.json())
	.then((json) => {
		if (json.print !== undefined) {
			console.log(json.print);
			saveData(json.print, "output.lua");
		} else {
			throw json.html;	// Lua script error has occurred
		}
	})
	.catch((error) => console.log(error));

Changing Data Type Of A Key

If I want to update the ImpactMultiplier, PunctureMultiplier, and SlashMultiplier keys to store table type from a number in Module:Stances/data (context being that physical damage bonuses do not apply to all hits of a single attack input which is why a table value is needed: to individually define physical damage multipliers to specific hits):

// Saves a file to local machine
var saveData = (function() {
	// Creating a temporary DOM element so we can 'click' on an element to download the file.
	// For obvious security reasons, JS running in browser environment does not have direct access to read/writes
	// to local storage. Browser is sandboxed to prevent arbitrary scripts causing damage to clients.
	var a = document.createElement("a");
	// document.body.appendChild(a);
	// a.style = "display: none";
	return function (data, fileName) {
		var json = JSON.stringify(data);
		// Unescaping escape characters
		json = json.replace(/\\n/gm, "\n").replace(/\\t/gm, "\t").replace(/\\"/gm, "\"")
		blob = new Blob([json], {type: "octet/stream"});
		blob = blob.slice(1, blob.size - 1); // Removing first and last quotation mark that designate json as a string
		url = window.URL.createObjectURL(blob);

		a.href = url;
		a.download = fileName;
		a.click();
		window.URL.revokeObjectURL(url);
	};
}() );

let origin = "https://warframe.fandom.com";
let path = "/api.php";
let params = {
	action: "scribunto-console",
	format: "json",
	title: "Module:Stances/data/dev",
	content: "",
	question: `
local StanceData = require('Module:Stances/data/dev')
local Serializer = require('Module:LuaSerializer')

for entryName, entry in pairs(StanceData) do
	for _, comboName in ipairs({ 'Block', 'Forward', 'Forward Block', 'Neutral', 'Aerial', 'Finisher', 'Heavy', 'Slide', 'Wall' }) do
		if (entry[comboName] ~= nil) then
			for i, attackInputEntry in pairs(entry[comboName].Attacks) do
				for _, physDmgBonusKey in ipairs({ 'SlashMultiplier', 'PunctureMultiplier', 'ImpactMultiplier' }) do
					local physDmgBonus = entry[comboName].Attacks[i][physDmgBonusKey]
					if (type(physDmgBonus) == 'number') then
						entry[comboName].Attacks[i][physDmgBonusKey] = { physDmgBonus }
					end 
				end
			end
		end
	end 
end

print(Serializer._serializeTable(StanceData))`,
	clear: 1
};

// [ ["action", "scribunto-console"], ["format","json"], ... ] to "action=scribunto-console&format=json&..."
let queryString = new URLSearchParams([ ...Object.entries(params) ]).toString();

let url = new URL(`${origin}${path}?${queryString}`);

fetch(url)
	.then((data) => data.json())
	.then((json) => {
		if (json.print !== undefined) {
			console.log(json.print);
			saveData(json.print, "output.lua");
		} else {
			throw json.html;	// Lua script error has occurred
		}
	})
	.catch((error) => console.log(error));

Horizontal Partitioning

In extreme cases, when a database gets too large or unwieldy to be edited on a single page, we may have to move data entries into separate, smaller subpages to improve performance. There are multiple ways to partition data, though which one is best to use depends on the context of the problem you are solving and how one chooses to access that data:

• By name, the letters (e.g. /data/A to /data/Z)
  • May fit for situations where data entry can be extrapolated from an article's name (i.e. {{PAGENAME}} on articles/templates or frame:getTitle() in Lua modules)
• By some developer-defined or player-defined category/classification system shared amongst entries of similar type (e.g. item type, gender, release date/mainline update, Mastery Rank lock)
  • See Module:Weapons/data for an example of this type of implementation on this wiki.

Limitations

As /data module subpages are not true databases, there are many limitations on how we store and access data:

• Lack of tools to add, remove, update, and manipulate data (CRUD). These have to be done manually by hand, editing data tables as if they were Lua code.
  • Therefore, there is no formal enforcement of data schemas on runtime. Data validation can only be done in a separate module and not in the moment these databases are changed. Syntax errors will be caught on edit submission however.
  • No easy way for aggregations or table joins.
  • No command-line interface.
• Maximum size of articles on the wiki are 2,048 kibibytes thus module pages are subjected to this restriction too.
  • Additional code formatting that are not part of data like spaces, tabs, and newlines will be counted towards this limit, though we still need them for human-readability.
• Everything is implemented as executable Lua programs. You have to execute the program in order to properly use the data. Data cannot be accessed as is.
  • Data can only be used in other Lua modules/programs without additional manipulation or conversion.
    • Note that other Lua modules cannot persist changes to /data subpages due to the nature of require() and mw.loadData(). All data is read-only.
  • The type of data stored are defined by Lua and the Scribunto extension: numbers (integers and floats), strings, tables, booleans, and nil
    • See https://www.mediawiki.org/wiki/Extension:Scribunto/Lua_reference_manual#Data_types
• Transactions are handled by the MediaWiki engine as if editors are editing articles, preventing multiple writes from occurring at the same time.
  • Edit conflicts can be resolved manually if another editor submits and edit while one is editing the page. MediaWiki will store a temporary copy of your changes in the editing interface so you can pick and choose what content to persist.
• There is are performance issues with hacking Lua tables as databases. These may be observed when articles exceed the allocated script time and memory usage.
  • Lua scripts are not running on a separate physical machine like a database in a typical client-server architecture. Instead, it is interpreted within PHP code with limited resources allocated to running the script.
    • See https://www.mediawiki.org/wiki/LuaSandbox

Benefits

There are very few benefits by storing data as code in the form of export Lua tables:

• Ability to add Lua-style comments without adding a designated field or metadata for annotations
  • Even JSON standard is designed to store purely data and lacks innate commenting support without implementing a parser that strips comment tokens before use
• No additional dependencies needed to perform simple Lua-based queries and manipulations in the debug console. In other words, one could write a simple Lua script to parse the data and output what you need into the console.

External Tools

• https://regex101.com/ - website for creating Regular expressions to help with data manipulation