WARFRAME Wiki
Advertisement
WARFRAME Wiki
CephalonSimaris
“Hunter, I have temporarily disabled that ability.”
This article contains JavaScript scripts that users can run locally in their browser's console or machine. As a warning, which goes for any scripts you copy/run from the Internet, MAKE SURE YOU UNDERSTAND THE CODE BEFORE RUNNING IT LOCALLY FOR YOUR OWN SECURITY! Contact an admin if you have any concerns or questions about a script.

This wiki contains many databases[1] and structured data regarding WARFRAME's contents and requires monthly updates to reflect on the accurate data of the game's contents. This article is meant to help editors understand how these databases interact with content on the rest of the wiki as well as how to update them. No programming experience nor wiki editing experience is needed to update our databases.

Contents of /data subpages can be downloaded to your local machine by adding the ?action=raw query string to the end of the URL of the respective subpage. These can be converted to other formats like JSON or CSV using external Lua libraries such as json-lua or cerial.lua. As a disclaimer, use the aforementioned libraries at your own risk as the wiki is not responsible for any damages that may occur.

Structured data stored on the wiki is not meant to reflect 100% on internal implementation or data of the game. Changes are made at editors' discretion to produce meaningful information that can be relayed to readers in accessible formats on the wiki and the wiki only. Data provided by DE through official means like Public Export will not necessarily contain the same information as the wiki and vice versa.

Last updated: Sun, 14 Jan 2024 02:10:26 +0000 (UTC) by User:Cephalon Scientia


List of Databases[]

Databases are denoted by /data which is the database subpage of a particular Lua module page.

How To Update[]

Generally, you do not have to be logged-in to edit /data subpages on the wiki. However, some highly dependent data require editors to be logged-in as an autoconfirmed user on the wiki to edit. This is to prevent vandalism and to uphold the integrity of our data. To be an autoconfirmed user, Fandom accounts must be at least four days old and had made 10 edits around the wiki before being able to edit these database pages. Otherwise, the contents of these pages will be publicly available in read-only mode.

To quickly find the right table entry to update, use your browser's CTRL + F hotkey or the wikitext source editor's search button in the toolbar under the "Advanced" tab.

If you make any syntax errors, the editor will alert you after you attempt to submit changes. Just navigate to the line number where it tells you an error occured and add the missing tokens before resubmitting. Common syntax errors include:

  • Missing a comma (,) at the end of key-value pairs
  • Forgetting a closing curly bracket (}) for tables
  • Invalid key name; if you want spaces or special characters in the key, make the key a string and put it inside square brackets like so: ["Key with spaces and colon:"] = "Some value"

Background[]

Our databases are in the form of Lua tables which have a similar syntax as JSON files, albeit tokens and delimiters are based on the Lua programming language.

Sample table:

TableKey = {
	KeyName = "Some string value",
    ["Another Key Name w/ Spaces"] = true,
    AnotherKeyName = { "Table values", "preferably", "be the same", "data type" },
    Integer = 2,
    Float = 3.1415926535897
}

These tables are parsed by the respective Lua module to be outputted to wikitext on pages that {{#invoke:}} a function from that particular module. For example, Module:Weapons/infobox uses data from Module:Weapons/data to generate the infobox for a particular weapon. Template:WeaponInfoboxAutomatic calls the buildInfobox() function in {{#invoke:Weapons/infobox|buildInfobox}} so any weapon article that uses/transcludes Template:WeaponInfoboxAutomatic will automatically generate the wikitext for the infobox as seen on the Braton article.

Possible Entries[]

There are limitations to the type of data that can be stored in these databases. Lua tables only support the following data types as values that are mapped to keys or as indexed elements:

  • Boolean (e.g. true or false)
  • Integer number (e.g. 10)
  • Float/decimal number (e.g. 2.5)
  • String (e.g. "Text")
  • Table (e.g. { "This", "is", "a", "table with", 6, "elements" })
  • Nil or null (e.g. nil)

Despite being able to be store functions in Lua tables, MediaWiki does not allow modules to read tables with functions as values so avoid adding these at all costs.

Data Pipeline[]

The wiki follows a three-tier architecture when handing data stored on the wiki's data stores.

How game data is transferred and processed on the wiki
Source Storage (Data Layer) Processing (Business Logic Layer) Use (Presentation Layer)
  • Data on game objects are manually audited and persisted to on-wiki /data submodules by volunteer wiki editors
  • Media assets uploaded to File namespace via Special:Upload by volunteer wiki editors (or via human-operated bots)
    • These are stored on actual databases/persistent storage + distributed via CDNs; wiki editors will never worry about this infrastructure
  • In the case of front-end scripts, sometimes data is stored within source code in arrays or JSON format
  • Lua modules and submodules access data via module imports and manipulate them in function calls
    • These are open-sourced and are likely unprotected for volunteer editors/developers to contribute to
  • Client-side scripts may make use of MediaWiki's Action API to fetch data from articles and process them for interactive elements
  • Templates are transcluded on articles which call a particular module's function to generate wikitext/HTML of data to display to readers
  • Very rarely a module's function is directly called on an article without the use of a template wrapper

Maintaining and Updating[]

The following are the most important databases to update in order of priority:

  1. Module:DropTables/data - WARFRAME Drop Tables
  2. Module:Weapons/data - used by weapon articles, Weapon Comparison, Template:WeaponNav, and tooltips
    • Lots of people rely on the comparison table for Disposition values (Riven Mods), comparing weapon stats, and determining what is "meta"
    • Module:Weapons/ppdata also needs to be updated when Module:Weapons/data is updated (need a manual edit to persist preprocessed data; we don't have a dedicated bot account/webhook for this)
  3. Module:Void/data - used by relic articles, Prime weapon/Warframe articles, Void Relic, and tooltips
    • These pages have high traffic whenver a new Prime Access or unvaulting is released since the in-game Codex is not very specific on drop chances of relics
  4. Module:Mods/data - used by mod articles and tooltips
  5. Module:Warframes/data - used by Warframe articles and tooltips
  6. Any other database related to content that was recently updated (e.g. Module:Baro/data if Baro Ki'Teer appears with new items)

Update Frequency[]

The frequency of updates are about once every three months (i.e. every quarter to match new Prime Access release) or every time a major mainline update is released by the developers. Minor updates can be made during downtime to fix values or to add additional data. Note that it is easier to add more data than to remove existing ones; wiki articles may already depend on old data to generate the pages' content so removing old/depreciated data is a more difficult task and would most likely require additional development time in the respective module pages.

New Content Update Frequency
Database Brief Description Approximate Frequency
Module:Version/data WARFRAME version builds Daily or weekly depending on hotfix/update schedule
Module:Baro/data Baro Ki'Teer visits 2 weeks, every other Friday when Baro Ki'Teer visits
Module:Weapons/data/primary Primary Weapons data 1-3 months, every other mainline update; Disposition update every 3 months with Prime Access
Module:Weapons/data/secondary Secondary Weapons data 1-3 months, every other mainline update; Disposition update every 3 months with Prime Access
Module:Weapons/data/melee Melee Weapons data 1-3 months, every other mainline update; Disposition update every 3 months with Prime Access
Module:Weapons/data/archwing Archgun and Archmelee Weapons data 1-3 months, every other mainline update; Disposition update every 3 months with Prime Access
Module:Weapons/data/companion Companion Weapons data Update as needed, when new companion weapons are released
Module:Weapons/data/railjack Railjack armaments data Update as needed, when new Railjack weapons are released
Module:Weapons/data/modular Modular Weapons data Update as needed, when new modular weapons are released; Disposition update every 3 months with Prime Access
Module:Weapons/data/misc Other Weapons data that doesn't fit in the other 7 partitions Update as needed
Module:Weapons/ppdata Weapons preprocessed data 1-3 months, every other mainline update; Disposition update every 3 months with Prime Access
Module:Blueprints/data Item blueprints 1-3 months, every other mainline update when a new weapon/item releases
Module:Resources/data Resources, currencies, foundry components 1-3 months, every other mainline update
Module:DropTables/data Drop Tables and rewards that are pulled from loot tables 1-3 months, every other mainline update
Module:Enemies/data Enemies data 1-3 months, every other mainline update
Module:Vendors/data Syndicate offerings, event shops, and NPC vendors 1-3 months, every other mainline update
Module:TennoGen/data TennoGen cosmetics 1-6 months, depending on TennoGen schedule
Module:Void/data Void Relic data 3 months for every Prime Access or every month for Prime Resurgence release
Module:Warframes/data Warframes data and related playable avatars (Archwings, Necramechs) 3 months, Prime Access release
Module:Ability/data Character abilities data 3 months, new Warframe release
Module:Cosmetics/data General Warframe Cosmetics Update as needed
Module:Codex/data Codex list Update as needed
Module:Arcane/data Arcane Enhancement data Update as needed, rarely updated
Module:Focus/data Focus data Update as needed, rarely updated
Module:Research/data Clan Research data Update as needed, rarely updated
Module:Modular/data Zaws, Kitguns, and other modular items Update as needed, rarely updated
Module:Conservation/data Conservation animals Update as needed, rarely updated
Module:DamageTypes/data Damage types and Status Effects Update as needed, rarely updated
Module:Missions/data Star Chart nodes data Update as needed, rarely updated
Module:Decorations/data Decorations data Update as needed, rarely updated
Module:GuaranteedRewards/data Rewards that are guaranteed to be rewarded to the player after completing some task (like Daily Tributes or after completing a quest). These are different from drop tables which rely on RNG. Update as needed, rarely updated

Formatting[]

  • Please use tabs for indentation instead of spaces.
  • Read the documentation above each /data subpage for sample formatting of key-value pairs. If there is no documentation written, try your best to follow the convention established in the Lua table entries

Validation[]

  • Some databases have a /data/validate subpage where editors can view to find errors in table entries.

For Developers[]

Database Design Guidelines[]

When designing new /data subpages of modules or adding new keys to existing /data tables:

  • Key names should be in TitleCase. If a key stores metadata and not data regarding the game's contents (i.e. meant to be used within modules and not presented to readers), prepend an underscore (_) in front of key name.
  • Keys should map to one value type and only that value type; do not mix and match value types otherwise it will cause more overhead in checking each individual value's types before performing a certain operation on that value. This also helps with script performance at runtime by reducing the number of conditional checks.
  • Tables should not be both array-like and dictionary-like. Stick to one usage of tables for consistent behavior when iterating over values.
  • If possible, map values to a key for faster access, especially when tables get extremely big. Internally, Lua implements table types as hash tables.[2]
    • Do not assume that keys represent data nor should they be treated as such. Keys are used to index a specific data table for faster access. It may be possible for an index to share the same value as a key-value pair inside the table, but this does not guarantee anything about the index as data. In most instances, the index will copy the Name key's value since it is more intuitive to look up data entries based on an item's in-game name.
  • Keep wikitext/HTML formatting outside of database entries as much as possible. In other words, prohibit (or at the very least limit) the use of wikitext syntax and HTML elements in string values. Databases should contain raw, unprocessed data. Any modifications to this data to display on the wiki should be done in main Lua modules and/or submodules. This is done to decouple data from business logic and user interface design like in a three-tier architecture.
  • Exported tables should not contain functions and/or metatables as values. These tables are strictly used to store data in the form of numbers, strings, booleans, and other tables.
  • Since most content in these databases will be presented to wiki readers, avoid storing string values in all capital casing or all lower casing. Use proper sentence casing or title casing instead.
    • Name localization may be an exception since DE stores localized strings in all caps for convention. We should store localized strings in their canonical form as much as possible to prevent loss of information.
  • Do not assume that an item's name is the same as its associated article's name on the wiki. Scenarios such as items with the same name, items with similar functions and/or type being represented under one generic article, items with not enough information to justify having its own page, formatting differences, or different localization may cause an article to have a different name than the item.
  • Tables should allow inversion using an inverted index to support different methods of indexing entries. For example:
    ["Abundant Mutation"] = {
    	BaseDrain = 6,
    	Image = "AbundantMutationMod.png",
    	Introduced = "27.5.4",
    	Link = "Abundant Mutation",
    	MaxRank = 3,
    	Name = "Abundant Mutation",
    	Polarity = "Zenurik",
    	Rarity = "Rare",
    	Tradable = true,
    	Transmutable = false,
    	Type = "Nidus" 
    }
    
    Can be manipulated to be indexed by Type:
    
    ["Nidus"] = {
    	BaseDrain = 6,
    	Image = "AbundantMutationMod.png",
    	Introduced = "27.5.4",
    	Link = "Abundant Mutation",
    	MaxRank = 3,
    	Name = "Abundant Mutation",
    	Polarity = "Zenurik",
    	Rarity = "Rare",
    	Tradable = true,
    	Transmutable = false,
    	Type = "Nidus" 
    }
    
    In the case that a particular inverted key has 
    multiple possible table entries, the inverted key 
    should instead map to a array-like table with each 
    element being a separate table entry.
    

Seeding New Databases[]

If creating new /data subpages, one could look to writing a script that parses Public Export or https://github.com/WFCD/warframe-items to automate the creation of new Lua tables based on existing JSON data. For Luafying data that exist on one or across many articles on the wiki, one could use Special:Export to export the contents of articles based on category or a manually created list to an XML format for extracting data programmatically. However, if what you are creating is completely new, then unfortunately, you may have to create the Lua tables by hand or write them down in a spreadsheet somewhere before converting it to Lua.

Accessing Data In Lua Modules[]

We provide two different interfaces to accessing data within Lua modules (and possibly third-party scripts via MediaWiki's Action API, see #Querying and Fetching Data for examples):

  1. Using Lua's built-in syntax for table accesses:
    • Index syntax like in most languages
      exampleTable[1] or exampleTable["keyName"]
    • Syntactic sugar with dot notation (doesn't work for numerical indexes)
      exampleTable.keyName
  2. Using Module:StatObject as an interface for raw data stored in /data subpages
    local StatObject = require('Module:StatObject')
    
    -- Define getter and format functions in StatObject.default table for 
    -- individual fields present in raw data or computed/derived fields
    StatObject.default = {
    	-- Getter functions:
    	-- Arrays of default values and format strings
    	key = { 'Default Value', '%.2f format' },
        
    	-- Can generate values/formatting with functions
    	-- 'val, ...' is the return from map[1]
    	key = { function(obj) end, function(self, val, ...) end },
        
    	-- If format is a table the return values will be passed to each function/format string
    	-- in the same order they are returned (nil is a pass-through)
    	key = {
    		function(obj) return a, b, c, d end,
    		{ function(self, val) return val end, '%s', nil, '%d', sep = '' }
    	},
        
    	-- Can omit second entry, can omit table
    	key = 'Default Value',
        
    	-- nil means default get (same as omitting)
    	key = nil,
        
        -- Add additional key-value pairs below to define raw getters and format getter functions for data
        -- To use the getters use StatObject.statRead(dataEntry, key) and StatObject.statFormat(dataEntry, key)
    }
    
    local ModData = mw.loadData('Module:Mods/data')		-- Importing sample database
    
    StatObject.statRead(ModData['Serration'], 'Name')	-- Example of getting the raw Name value of Serration mod entry
    StatObject.statFormat(ModData['Serration'], 'Name')	-- Example of getting the formatted Name value of Serration mod entry
    

Querying and Fetching Data[]

If you want to query these databases, you can write a Lua script to parse and return the data you want in Special:ApiSandbox. MediaWiki's Action API using action=scribunto-console can be used for this purpose.

For example, if one wants to find the names of secondary mods that have the Vazarin (V or Vazarin Pol) polarity, the following snippet can be passed into the question parameter:

local ModData = require('Module:Mods/data').Mods
local results = {}
for _, modEntry in pairs(ModData) do
    if (modEntry.Polarity == 'Vazarin' and modEntry.Type == 'Pistol') then
        table.insert(results, modEntry.Name)
    end
end
print(table.concat(results, ', '))

A sample of the above query in ApiSandbox: https://warframe.fandom.com/wiki/Special:ApiSandbox#action=scribunto-console&format=json&title=Module%3AMods%2Fdata&content=&question=local%20ModData%20%3D%20require('Module%3AMods%2Fdata').Mods%0Alocal%20results%20%3D%20%7B%7D%0Afor%20_%2C%20modEntry%20in%20pairs(ModData)%20do%0Aif%20(modEntry.Polarity%20%3D%3D%20'Vazarin'%20and%20modEntry.Type%20%3D%3D%20'Pistol')%20then%0Atable.insert(results%2C%20modEntry.Name)%0Aend%0Aend%0Aprint(table.concat(results%2C%20'%2C%20'))&clear=1

Output to CSV or JSON[]

The wiki has forked some conversion libraries Module:CSV and Module:JSON for developers' convenience when trying to output the data in CSV or JSON format.

For example, if one wants to get the contents of Module:Mods/data in JSON:

local ModData = require('Module:Mods/data').Mods
local json = require('Module:JSON')
print(json.stringify(ModData))

A sample of the above query in ApiSandbox: https://warframe.fandom.com/wiki/Special:ApiSandbox#action=scribunto-console&format=json&title=Module%3AMods%2Fdata&content=&question=local%20ModData%20%3D%20require('Module%3AMods%2Fdata').Mods%0Alocal%20json%20%3D%20require('Module%3AJSON')%0Aprint(json.stringify(ModData))&clear=1


Sample JS using Action API:

let origin = "https://warframe.fandom.com";
let path = "/api.php";
let params = {
	action: "scribunto-console",
	format: "json",
	title: "Module:Mods/data",
	content: "",
	question: `
local ModData = require('Module:Mods/data').Mods
local json = require('Module:JSON')
print(json.stringify(ModData))`,
	clear: 1
};

// [ ["action", "scribunto-console"], ["format","json"], ... ] to "action=scribunto-console&format=json&..."
let queryString = new URLSearchParams([ ...Object.entries(params) ]).toString();

let url = new URL(`${origin}${path}?${queryString}`);

fetch(url)
	.then((data) => data.json())
		.then((json) => {
		if (json.print !== undefined) {	// Replace with json.return if you want to get content from return statement
			console.log(JSON.parse(json.print));
		} else {
			throw json.html;	// Lua script error has occurred
		}
	})
	.catch((error) => console.log(error));

Updating Databases via API[]

Here are some sample scripts for reference:

Adding New/Missing Keys[]

For example, if I want to add missing keys in Module:Weapons/data/melee the following script can be run to save the edited database onto your local machine which can be persisted to the database via a manual copy and paste[3]:

// Saves a file to local machine
var saveData = (function() {
	// Creating a temporary DOM element so we can 'click' on an element to download the file.
	// For obvious security reasons, JS running in browser environment does not have direct access to read/writes
	// to local storage. Browser is sandboxed to prevent arbitrary scripts causing damage to clients.
	var a = document.createElement("a");
	// document.body.appendChild(a);
	// a.style = "display: none";
	return function (data, fileName) {
		var json = JSON.stringify(data);
		// Unescaping escape characters
		json = json.replace(/\\n/gm, "\n").replace(/\\t/gm, "\t").replace(/\\"/gm, "\"")
		blob = new Blob([json], {type: "octet/stream"});
		blob = blob.slice(1, blob.size - 1); // Removing first and last quotation mark that designate json as a string
		url = window.URL.createObjectURL(blob);

		a.href = url;
		a.download = fileName;
		a.click();
		window.URL.revokeObjectURL(url);
	};
}() );

let origin = "https://warframe.fandom.com";
let path = "/api.php";
let params = {
	action: "scribunto-console",
	format: "json",
	title: "Module:DropTables/data",
	content: "",
	question: `
local DropData = require('Module:Weapons/data/melee')
local serialize = require('Module:LuaSerializer')
for _, t in pairs(DropData) do
	if (t.Family == nil) then
		t.Family = t.Name
	end
end
print(serialize._serializeTable(DropData))`,
	clear: 1
};

// [ ["action", "scribunto-console"], ["format","json"], ... ] to "action=scribunto-console&format=json&..."
let queryString = new URLSearchParams([ ...Object.entries(params) ]).toString();

let url = new URL(`${origin}${path}?${queryString}`);

fetch(url)
	.then((data) => data.json())
	.then((json) => {
		if (json.print !== undefined) {
			console.log(json.print);
			saveData(json.print, "output.lua");
		} else {
			throw json.html;	// Lua script error has occurred
		}
	})
	.catch((error) => console.log(error));

Updating Table Values[]

If I want to update Module:DropTables/data entries so that all item drop entries in each particular drop table contain a string of the item type as their second element based on the name of the drop table (e.g. mod drop tables have items of type "Mod"). Note that some items in drop tables do not match the drop table type which is why an explicit value is needed for manual exceptions (e.g. you can technically have resources that drop from the mod drop table; there is loose enforcement on item types in a drop table).

// Saves a file to local machine
var saveData = (function() {
	// Creating a temporary DOM element so we can 'click' on an element to download the file.
	// For obvious security reasons, JS running in browser environment does not have direct access to read/writes
	// to local storage. Browser is sandboxed to prevent arbitrary scripts causing damage to clients.
	var a = document.createElement("a");
	// document.body.appendChild(a);
	// a.style = "display: none";
	return function (data, fileName) {
		var json = JSON.stringify(data);
		// Unescaping escape characters
		json = json.replace(/\\n/gm, "\n").replace(/\\t/gm, "\t").replace(/\\"/gm, "\"")
		blob = new Blob([json], {type: "octet/stream"});
		blob = blob.slice(1, blob.size - 1); // Removing first and last quotation mark that designate json as a string
		url = window.URL.createObjectURL(blob);

		a.href = url;
		a.download = fileName;
		a.click();
		window.URL.revokeObjectURL(url);
	};
}() );

let origin = "https://warframe.fandom.com";
let path = "/api.php";
let params = {
	action: "scribunto-console",
	format: "json",
	title: "Module:DropTables/data",
	content: "",
	question: `
local DropData = require('Module:DropTables/data')
local Serializer = require('Module:LuaSerializer')

for _, collection in ipairs({ 'Containers', 'Enemies' }) do
	for entryName, entry in pairs(DropData[collection] or {}) do
		for _, dropTableName in ipairs({ 'Blueprints', 'Items', 'Mods', 'Relics', 'Resources', 'Sigils'}) do
			for i, dropEntry in ipairs(entry[dropTableName] or {}) do
				-- Remove 's' in dropTableName to get item type
				table.insert(dropEntry, 2, dropTableName:sub(1, -2))
			end
		end 
	end
end

print(Serializer._serializeTable({ Containers = DropData.Containers, Enemies = DropData.Enemies }))`,
	clear: 1
};

// [ ["action", "scribunto-console"], ["format","json"], ... ] to "action=scribunto-console&format=json&..."
let queryString = new URLSearchParams([ ...Object.entries(params) ]).toString();

let url = new URL(`${origin}${path}?${queryString}`);

fetch(url)
	.then((data) => data.json())
	.then((json) => {
		if (json.print !== undefined) {
			console.log(json.print);
			saveData(json.print, "output.lua");
		} else {
			throw json.html;	// Lua script error has occurred
		}
	})
	.catch((error) => console.log(error));

Changing Data Type Of A Key[]

If I want to update the ImpactMultiplier, PunctureMultiplier, and SlashMultiplier keys to store table type from a number in Module:Stances/data (context being that physical damage bonuses do not apply to all hits of a single attack input which is why a table value is needed: to individually define physical damage multipliers to specific hits):

// Saves a file to local machine
var saveData = (function() {
	// Creating a temporary DOM element so we can 'click' on an element to download the file.
	// For obvious security reasons, JS running in browser environment does not have direct access to read/writes
	// to local storage. Browser is sandboxed to prevent arbitrary scripts causing damage to clients.
	var a = document.createElement("a");
	// document.body.appendChild(a);
	// a.style = "display: none";
	return function (data, fileName) {
		var json = JSON.stringify(data);
		// Unescaping escape characters
		json = json.replace(/\\n/gm, "\n").replace(/\\t/gm, "\t").replace(/\\"/gm, "\"")
		blob = new Blob([json], {type: "octet/stream"});
		blob = blob.slice(1, blob.size - 1); // Removing first and last quotation mark that designate json as a string
		url = window.URL.createObjectURL(blob);

		a.href = url;
		a.download = fileName;
		a.click();
		window.URL.revokeObjectURL(url);
	};
}() );

let origin = "https://warframe.fandom.com";
let path = "/api.php";
let params = {
	action: "scribunto-console",
	format: "json",
	title: "Module:Stances/data/dev",
	content: "",
	question: `
local StanceData = require('Module:Stances/data/dev')
local Serializer = require('Module:LuaSerializer')

for entryName, entry in pairs(StanceData) do
	for _, comboName in ipairs({ 'Block', 'Forward', 'Forward Block', 'Neutral', 'Aerial', 'Finisher', 'Heavy', 'Slide', 'Wall' }) do
		if (entry[comboName] ~= nil) then
			for i, attackInputEntry in pairs(entry[comboName].Attacks) do
				for _, physDmgBonusKey in ipairs({ 'SlashMultiplier', 'PunctureMultiplier', 'ImpactMultiplier' }) do
					local physDmgBonus = entry[comboName].Attacks[i][physDmgBonusKey]
					if (type(physDmgBonus) == 'number') then
						entry[comboName].Attacks[i][physDmgBonusKey] = { physDmgBonus }
					end 
				end
			end
		end
	end 
end

print(Serializer._serializeTable(StanceData))`,
	clear: 1
};

// [ ["action", "scribunto-console"], ["format","json"], ... ] to "action=scribunto-console&format=json&..."
let queryString = new URLSearchParams([ ...Object.entries(params) ]).toString();

let url = new URL(`${origin}${path}?${queryString}`);

fetch(url)
	.then((data) => data.json())
	.then((json) => {
		if (json.print !== undefined) {
			console.log(json.print);
			saveData(json.print, "output.lua");
		} else {
			throw json.html;	// Lua script error has occurred
		}
	})
	.catch((error) => console.log(error));

Horizontal Partitioning[]

In extreme cases, when a database gets too large or unwieldy to be edited on a single page, we may have to move data entries into separate, smaller subpages to improve performance. There are multiple ways to partition data, though which one is best to use depends on the context of the problem you are solving and how one chooses to access that data:

  • By name, the letters (e.g. /data/A to /data/Z)
    • May fit for situations where data entry can be extrapolated from an article's name (i.e. {{PAGENAME}} on articles/templates or frame:getTitle() in Lua modules)
  • By some developer-defined or player-defined category/classification system shared amongst entries of similar type (e.g. item type, gender, release date/mainline update, Mastery Rank lock)

Limitations[]

As /data module subpages are not true databases, there are many limitations on how we store and access data:

  • Lack of tools to add, remove, update, and manipulate data (CRUD). These have to be done manually by hand, editing data tables as if they were Lua code.
    • Therefore, there is no formal enforcement of data schemas on runtime. Data validation can only be done in a separate module and not in the moment these databases are changed. Syntax errors will be caught on edit submission however.
    • No easy way for aggregations or table joins.
    • No command-line interface.
  • Maximum size of articles on the wiki are 2,048 kibibytes thus module pages are subjected to this restriction too.
    • Additional code formatting that are not part of data like spaces, tabs, and newlines will be counted towards this limit, though we still need them for human-readability.
  • Everything is implemented as executable Lua programs. You have to execute the program in order to properly use the data. Data cannot be accessed as is.
    • Data can only be used in other Lua modules/programs without additional manipulation or conversion.
      • Note that other Lua modules cannot persist changes to /data subpages due to the nature of require() and mw.loadData(). All data is read-only.
    • The type of data stored are defined by Lua and the Scribunto extension: numbers (integers and floats), strings, tables, booleans, and nil
  • Transactions are handled by the MediaWiki engine as if editors are editing articles, preventing multiple writes from occurring at the same time.
    • Edit conflicts can be resolved manually if another editor submits and edit while one is editing the page. MediaWiki will store a temporary copy of your changes in the editing interface so you can pick and choose what content to persist.
  • There is are performance issues with hacking Lua tables as databases. These may be observed when articles exceed the allocated script time and memory usage.
    • Lua scripts are not running on a separate physical machine like a database in a typical client-server architecture. Instead, it is interpreted within PHP code with limited resources allocated to running the script.

Benefits[]

There are very few benefits by storing data as code in the form of export Lua tables:

  • Ability to add Lua-style comments without adding a designated field or metadata for annotations
    • Even JSON standard is designed to store purely data and lacks innate commenting support without implementing a parser that strips comment tokens before use[4][5]
  • No additional dependencies needed to perform simple Lua-based queries and manipulations in the debug console. In other words, one could write a simple Lua script to parse the data and output what you need into the console.

External Tools[]

  1. As a footnote, for simplicity's sake, we use the term "database" to refer to data stored in Lua tables on /data subpages of Lua modules in a NoSQL-like format. These are not true databases in the traditional sense with advanced features like indexing, data validation, and stored procedures. In reality, these are executable programs that return Lua tables with data. These can also be thought as virtual in-memory databases that share the same memory as Lua programs.
  2. https://www.lua.org/doc/jucs05.pdf
  3. https://stackoverflow.com/questions/43135852/javascript-export-to-text-file
  4. Gundlach, Michael (2008, October 28). Can comments be used in JSON?. StackOverflow. Accessed 2022-11-15. Archived from the original on 2022-11-15.
  5. Crockford, Douglas (2012, April 30). I removed comments from JSON because I saw people were using them to hold parsing directives, a practice which would have destroyed interoperability. I know that the lack of comments makes some people sad, but it shouldn't.. Google+. Accessed 2022-11-15. Archived from the original on 2015-01-05.
Advertisement