{"id":6791,"date":"2015-03-06T16:54:54","date_gmt":"2015-03-06T21:54:54","guid":{"rendered":"http:\/\/www.iri.com\/blog\/?p=6791"},"modified":"2019-04-01T09:56:02","modified_gmt":"2019-04-01T13:56:02","slug":"creating-test-data-for-mongodb","status":"publish","type":"post","link":"https:\/\/www.iri.com\/blog\/vldb-operations\/creating-test-data-for-mongodb\/","title":{"rendered":"Creating Test Data for MongoDB"},"content":{"rendered":"<p class=\"western\" lang=\"zxx\"><em>Introduction: This example demonstrates<strong> an older method<\/strong> of using IRI RowGen to generate and populate large or complex collection prototypes for testing or system capacity using flat files. As you will read, RowGen would create the necessary test data and create a CSV file that would be loaded into MongoDB using the Mongo Import Utility. <\/em><\/p>\n<p class=\"western\" lang=\"zxx\"><em><strong>2019 Update: IRI now also offers<\/strong>\u00a0JSON and direct driver support to move data between MongoDB collections and <a href=\"https:\/\/www.iri.com\/products\/cosort\/sortcl\">SortCL<\/a>-compatible IRI software products like RowGen or FieldShield. This means you can use RowGen to generate test <a href=\"https:\/\/www.iri.com\/solutions\/data-and-database-migration\/file-conversion\/json\">JSON files<\/a> for import into MongoDB (not unlike the method shown below in this article), or use FieldShield to <a href=\"https:\/\/www.iri.com\/blog\/data-protection\/native-mongodb-masking-voracity\/\">mask data in Mongo<\/a> tables into test targets.<\/em><\/p>\n<p class=\"western\" lang=\"zxx\"><em>Note that both FieldShield and RowGen are included in the <a href=\"https:\/\/www.iri.com\/products\/voracity\">IRI Voracity<\/a>\u00a0data management platform, which offers\u00a0<a href=\"https:\/\/www.iri.com\/solutions\/test-data#techniques\">four ways<\/a> to create test data.\u00a0\u00a0<\/em><\/p>\n<hr \/>\n<p class=\"western\" lang=\"zxx\">Although MongoDB is a fine\u00a0cross-platform, document-oriented NoSQL database, it has no convenient way to generate and populate large\u00a0or\u00a0complex collection prototypes\u00a0that can be used to\u00a0test queries or plan capacity. This article explains how to create test data MongoDB can use via\u00a0<a href=\"http:\/\/www.iri.com\/products\/rowgen\" target=\"_blank\" rel=\"noopener\">IRI RowGen<\/a>,\u00a0specifying the parameters for a synthetic, but realistic, CSV file that MongoDB can\u00a0import for functional and performance testing.<\/p>\n<p class=\"western\" lang=\"zxx\">You must first consider the\u00a0structure and content of the test data for\u00a0your collection (MongoDB table) needs. See <a href=\"http:\/\/www.iri.com\/blog\/test-data\/test-data-management-test-data-needs-assessment\/\" target=\"_blank\" rel=\"noopener\">this<\/a>\u00a0article\u00a0for typical planning considerations.<\/p>\n<p class=\"western\" lang=\"zxx\">In the\u00a0example, we know that our collection will be made up of customers who all have <strong>Usernames<\/strong>, <strong>First and Last Names<\/strong>, <strong>Email Addresses<\/strong>, and <strong>Credit Card Numbers<\/strong>.<\/p>\n<p lang=\"zxx\">To\u00a0create our test data, we must first generate some set files. A set file is a list of one or more tab-delimited values that may already exist, or need to be generated manually or automatically from database columns through the &#8216;Generate New Set File&#8217; wizard in IRI RowGen.<\/p>\n<p lang=\"zxx\"><em><strong>Generating Names<\/strong><\/em><\/p>\n<p lang=\"zxx\">1) Create a compound data value\u00a0(first and last names combined) job script\u00a0named &#8220;CreateNamesSet.rcl&#8221; that RowGen can execute to produce a set file; call the\u00a0output &#8220;User.set&#8221; because these names will also be used as the basis for\u00a0our usernames.<\/p>\n<p lang=\"zxx\">2) Create three fields to be generated in Names.set: last name, tab separator, and first name. Name the first field &#8220;LastName&#8221; and\u00a0choose\u00a0the method that will\u00a0select\u00a0values from an IRI-provided set file called &#8220;names_last.set&#8221;. Add the literal value &#8220;\\t&#8221; to add a tab separator, and then repeat the process used for\u00a0LastName and FirstName values using names_first.set.<\/p>\n<p lang=\"zxx\">3) Run CreateNamesSet.rcl with RowGen, either on the command line or from the IRI Workbench GUI, to produce the tab-delimited User.set\u00a0file\u00a0of first and last names, which will be used in both the generation of usernames and in the final test file build that populates our prototype collection.<\/p>\n<p lang=\"zxx\"><a href=\"http:\/\/www.iri.com\/blog\/wp-content\/uploads\/2015\/02\/User1.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-6814 size-full\" src=\"http:\/\/www.iri.com\/blog\/wp-content\/uploads\/2015\/02\/User1-e1425678943171.png\" alt=\"Mongo_RowGen_User\" width=\"600\" height=\"555\" \/><\/a><\/p>\n<p lang=\"zxx\"><em><strong>Generating Usernames<\/strong><\/em><\/p>\n<p lang=\"zxx\">For Usernames, we will create a set file that utilizes\u00a0the Users.set file generated above. Usernames for this example will combine\u00a0last name, first initial, and a randomly generated number between 100 and 999.<\/p>\n<p lang=\"zxx\">1) Create a new RowGen job script\u00a0with the Compound Data Wizard, call it &#8220;CreateUsernamesSet.rcl&#8221;, and name the output set file &#8220;Usernames.set&#8221;.<\/p>\n<p lang=\"zxx\">2) Build\u00a0compound username values with\u00a0three components named Part1, Part2, and Part3.<\/p>\n<p lang=\"zxx\">3) For Part1, choose\u00a0the method that will\u00a0select\u00a0values from (browse to) the previously-generated User.set file, and specify <span style=\"line-height: 1.5;\">&#8216;ALL&#8217; for the selection type to maintain the association between users, usernames, and email addresses. Set the size to 5.<\/span><\/p>\n<p lang=\"zxx\">4)\u00a0<span style=\"line-height: 1.5;\">For Part2 repeat the process used for Part1, except for Selection type, select &#8216;Row&#8217; and set Column Index to 2. \u00a0Set the size to 1. This guarantees all the last names will be used in the generation, and that the first letter of the first name in the same row is appended to the user name. <\/span><\/p>\n<p lang=\"zxx\"><span style=\"line-height: 1.5;\">5) For Part3<\/span><span style=\"line-height: 1.5;\">, specify the\u00a0generation of a n<\/span><span style=\"line-height: 1.5;\">umeric value between 100 <\/span><span style=\"line-height: 1.5;\">and\u00a0<\/span><span style=\"line-height: 1.5;\">999 to suffix a random integer with each username.<\/span><\/p>\n<p lang=\"zxx\">Upon execution\u00a0of CreateUsernamesSet.rcl,\u00a0we see that each username\u00a0contains\u00a0the first five letters of their last name, then their first initial, then a random 3-digit number:<\/p>\n<p lang=\"zxx\"><a href=\"http:\/\/www.iri.com\/blog\/wp-content\/uploads\/2015\/02\/Username.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-6815 size-full\" src=\"http:\/\/www.iri.com\/blog\/wp-content\/uploads\/2015\/02\/Username-e1425678960862.png\" alt=\"Mongo_RowGen_Username\" width=\"600\" height=\"565\" \/><\/a><\/p>\n<p lang=\"zxx\"><em><strong>Generating\u00a0Emails<\/strong><\/em><\/p>\n<p lang=\"zxx\">Next we will create an\u00a0email set file that appends\u00a0the username values with\u00a0randomly-selected domain names. Because some email services are more popular than others, we will also create a weighting system to reflect a higher frequency of\u00a0yahoo and gmail domains.<\/p>\n<p lang=\"zxx\">1) Run RowGen&#8217;s &#8216;New Custom Test Data&#8217; job wizard to create a job called &#8220;CreateEmailsSet&#8221; that produces a set file called &#8220;Emails.set&#8221;.<\/p>\n<p lang=\"zxx\">2) Produce the username part of the email. In the Test Data Definition dialog, click New Field, and rename the first field Usernames. Double-click on it to launch the Generation Field dialog and &#8220;Define &#8230;&#8221; its Set file as Usernames.set. Set the size to 9 and click OK.<\/p>\n<p lang=\"zxx\">3) Produce the domain part of the email (which includes the @ symbol). In the Layout Fields dialog, click New Field and rename it to &#8220;address&#8221; and double-click on it. In the Generation Field dialog, specify a &#8221; ,&#8221; with a position of 10 and a size of 20. In the Data Generation \/ Data Distribution section below, click &#8220;Define &#8230;&#8221; to name a new data distribution of items &#8220;WeightedEmails&#8221;.<\/p>\n<p lang=\"zxx\">4) In the New Distribution Wizard, chose &#8216;Weighted Distribution of Items&#8217; and enter these\u00a0items into the ratio and literal text boxes respectively, then add each to the list.<\/p>\n<pre>(32 | @gmail.com), (32 | @yahoo.com),\u00a0(2 | @ibm.com),\u00a0(4 | @msn.com), (2 | @ymail.com),\u00a0(2 | @inmail.com), (2 | @cnet.net),\u00a0(2 | @chase.org),\u00a0(1 | @iri.com), (1 | @gdic.com),\u00a0(1 | @aci.com), (2 | @oracle.net),\u00a0(1 | @gmx.org),\u00a0(4 | @aol.com), (2 | @inbox.com),\u00a0(2 | @hushmail.com), (2 | @outlook.com),\u00a0(2 | @zoho.com), (2 | @yandex.net),\u00a0(2 | @mail.com)<\/pre>\n<p>After you enter these values, click Next in the original wizard to move into\u00a0the Data Targets dialog. Use &#8220;Add Data Target &#8230;&#8221; to specify the output file &#8220;Email.set&#8221;. This will also be used at collection-build time.<\/p>\n<blockquote>\n<p lang=\"zxx\"><a href=\"http:\/\/www.iri.com\/blog\/wp-content\/uploads\/2015\/02\/Emails.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-6816 size-full\" src=\"http:\/\/www.iri.com\/blog\/wp-content\/uploads\/2015\/02\/Emails-e1425678987850.png\" alt=\"Mongo_RowGen_Emails\" width=\"600\" height=\"463\" \/><\/a><\/p>\n<p lang=\"zxx\"><em>The email we set the highest weights for (gmail and yahoo) show up most frequently, with others showing up periodically.<\/em><\/p>\n<\/blockquote>\n<p lang=\"zxx\"><em><strong>Generating Credit Card Numbers<\/strong><\/em><\/p>\n<p lang=\"zxx\">Lastly, we will create\u00a0computationally valid\u00a0card numbers in\u00a0the format XXXX-XXXX-XXXX-XXXX. The first four digits reflect actual\u00a0Issue Identifier Numbers (IIN) of various\u00a0credit card companies, and\u00a0the last digit verifies\u00a0the cards&#8217; authenticity.<\/p>\n<p lang=\"zxx\">To do this, create and run a new (empty) job. Call it &#8220;CreateCCNSet.rcl&#8221; (or .scl), and populate it with the script below to create &#8220;CCN.set&#8221;. The \/INCOLLECT value in RowGen scripts determines the number of rows generated.<\/p>\n<p lang=\"zxx\"><a href=\"http:\/\/www.iri.com\/blog\/wp-content\/uploads\/2015\/03\/CreditCardScript.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-6830 size-full\" src=\"http:\/\/www.iri.com\/blog\/wp-content\/uploads\/2015\/03\/CreditCardScript-e1425678999837.png\" alt=\"Mongo_RowGen_CreditCardScript\" width=\"600\" height=\"377\" \/><\/a><\/p>\n<p lang=\"zxx\"><em>RowGen&#8217;s purpose-built CCN generation function,\u00a0ccn_gen(&#8220;ANY, &#8220;-&#8220;) is called to populate this field. Note similar functions exist for <a href=\"http:\/\/www.iri.com\/blog\/test-data\/united-states-social-security-number\/\" target=\"_blank\" rel=\"noopener\">US<\/a> and <a href=\"http:\/\/www.iri.com\/blog\/test-data\/korea-social-security-number\/\" target=\"_blank\" rel=\"noopener\">Korean<\/a> social security numbers, and the national IDs of <a href=\"http:\/\/www.iri.com\/blog\/test-data\/generating-test-nid-data-italy-fiscal-codes\/\" target=\"_blank\" rel=\"noopener\">Italy<\/a> and <a href=\"http:\/\/www.iri.com\/blog\/test-data\/netherlands-social-fiscal\/\" target=\"_blank\" rel=\"noopener\">The Netherlands<\/a>.<\/em><\/p>\n<p lang=\"zxx\"><span style=\"text-decoration: underline;\"><strong>Creating the Final Test\u00a0File<\/strong><\/span><\/p>\n<p lang=\"zxx\">With all set files built, it is time to\u00a0use them in the test\u00a0CSV file we&#8217;ll create and export\u00a0to a MongoDB collection.<\/p>\n<p lang=\"zxx\">1) Run RowGen&#8217;s &#8216;New Custom Test Data&#8217; job wizard to create a job called &#8220;CreateMongoUserData.rcl&#8221; that will generate the\u00a0Customers.csv file,\u00a0 the file we will then\u00a0export to MongoDB.<\/p>\n<p lang=\"zxx\">2)\u00a0Click &#8220;Layout Fields &#8230;&#8221; to enter the Layout Fields Dialog. Click New Field and rename the first field to Usernames.\u00a0Double-click on it to launch the Generation Field dialog and \u201cDefine \u2026\u201d its Set file as Usernames.set; then select ALL for its selection type.<\/p>\n<p lang=\"zxx\">3)\u00a0Click New Field and rename the second\u00a0field to LastNames.\u00a0Double-click on it to launch the Generation Field dialog and \u201cDefine \u2026\u201d its Set file as Users.set; then select ALL for its selection type.<\/p>\n<p lang=\"zxx\">4)\u00a0Click New Field and rename the third\u00a0field to FirstNames.\u00a0Double-click on it to launch the Generation Field dialog and \u201cDefine \u2026\u201d its Set file as Users.set; then select\u00a0ROWS for its selection type and set the column index to 2.<\/p>\n<p lang=\"zxx\">5)\u00a0Click New Field and rename the\u00a0fourth\u00a0field to Email.\u00a0Double-click on it to launch the Generation Field dialog and \u201cDefine \u2026\u201d its Set file as Emails.set; then select ALL for its selection type.<\/p>\n<p lang=\"zxx\">6)\u00a0Click New Field and rename the fifth\u00a0field to CreditCardNumbers.\u00a0Double-click on it to launch the Generation Field dialog and \u201cDefine \u2026\u201d its Set file as CCN.set; then select ALL for its selection type.<\/p>\n<p lang=\"zxx\">7)\u00a0After you enter these values, click Next in the original wizard to move into\u00a0the Data Targets dialog. Use \u201cAdd Data Target \u2026\u201d to specify the output file Customers.csv; then run the script\u00a0in the Workbench or on the command line to generate that file:<\/p>\n<pre lang=\"zxx\">rowgen \/spec=CreateMongoUserData.rcl<\/pre>\n<p lang=\"zxx\"><a href=\"http:\/\/www.iri.com\/blog\/wp-content\/uploads\/2015\/02\/FinalScript3.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-6818 size-full\" src=\"http:\/\/www.iri.com\/blog\/wp-content\/uploads\/2015\/02\/FinalScript3-e1425679012191.png\" alt=\"Mongo_RowGen_FinalScript\" width=\"600\" height=\"441\" \/><\/a><\/p>\n<p lang=\"zxx\">Note that RowGen, in addition to producing this CSV file at runtime, could have also produced multiple, other file, database, formatted-report, named-pipe, procedural, and even real-time BIRT display, with fields from the generated test data, all at the same time.<\/p>\n<p lang=\"zxx\"><span style=\"text-decoration: underline;\"><b>Importing to MongoDB<\/b><\/span><\/p>\n<p lang=\"zxx\">To import the CSV file into your Mongo Database, call the &#8216;mongoimport utility&#8217; and run the following command:<\/p>\n<pre>--db &lt;Database Name&gt; --collection &lt;Collection Name&gt; --type csv --fields &lt;fieldname1,fieldname2,...&gt; --file &lt;File path to the CSV file to import&gt;<\/pre>\n<p lang=\"zxx\"><a href=\"http:\/\/www.iri.com\/blog\/wp-content\/uploads\/2015\/02\/Import3.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-6819 size-full\" src=\"http:\/\/www.iri.com\/blog\/wp-content\/uploads\/2015\/02\/Import3-e1425679024849.png\" alt=\"Mongo_RowGen_Import\" width=\"500\" height=\"255\" \/><\/a><\/p>\n<p lang=\"zxx\">Here are the records in the test collection (shown with <a href=\"http:\/\/www.mongovue.com\/\" target=\"_blank\" rel=\"noopener\">MongoVUE<\/a>), which MongoDB will automatically index with generated ID values for each entry:<\/p>\n<p lang=\"zxx\"><a href=\"http:\/\/www.iri.com\/blog\/wp-content\/uploads\/2015\/02\/InfoInMongo2.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-6821 size-full\" src=\"http:\/\/www.iri.com\/blog\/wp-content\/uploads\/2015\/02\/InfoInMongo2-e1425679035737.png\" alt=\"Mongo_RowGen_InfoInMongo\" width=\"600\" height=\"491\" \/><\/a><\/p>\n<p class=\"western\" lang=\"zxx\">MongoDB assigns a unique ID value to each collection entry.<\/p>\n<p>You can also load test data directly into the Mongo Database using <span style=\"background-color: #ffffff; color: #111111;\">Progress Software\u2019s DataDirect ODBC driver for MongoDB. <\/span>Before running the RowGen job in the Workbench, I had an empty collection called CUSTOMERS_CNN in MYDB to receive the data.<\/p>\n<p>I ran the job first using <span style=\"font-style: italic;\">stdout, <\/span>to preview my test data in the console window:<\/p>\n<p><a title=\"RowGen MongoDB blog article update 0.jpg\" href=\"http:\/\/www.iri.com\/blog\/wp-content\/uploads\/2016\/02\/RowGen-MongoDB-blog-article-update-0.jpg\" rel=\"prettyPhoto\"><img decoding=\"async\" class=\"aligncenter\" style=\"width: 624px; height: 468px;\" src=\"http:\/\/www.iri.com\/blog\/wp-content\/uploads\/2016\/02\/t_RowGen-MongoDB-blog-article-update-0.jpg\" alt=\"\" \/><\/a><\/p>\n<p>After executing the script in the Workbench, I can now see my data using the Data Source Explorer, and the DataDirect JDBC driver.<\/p>\n<p><a title=\"RowGen MongoDB blog article update 1.jpg\" href=\"http:\/\/www.iri.com\/blog\/wp-content\/uploads\/2016\/02\/RowGen-MongoDB-blog-article-update-1.jpg\" rel=\"prettyPhoto\"><img decoding=\"async\" class=\"aligncenter\" style=\"width: 624px; height: 468px;\" src=\"http:\/\/www.iri.com\/blog\/wp-content\/uploads\/2016\/02\/t_RowGen-MongoDB-blog-article-update-1.jpg\" alt=\"\" \/><\/a><\/p>\n<p class=\"western\" lang=\"zxx\">For more information on the generation options available, see the\u00a0<strong>Test File Targets<\/strong> section at:\u00a0<a href=\"http:\/\/www.iri.com\/products\/rowgen\/technical-details\" target=\"_blank\" rel=\"noopener\">http:\/\/www.iri.com\/products\/rowgen\/technical-details<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Introduction: This example demonstrates an older method of using IRI RowGen to generate and populate large or complex collection prototypes for testing or system capacity using flat files. As you will read, RowGen would create the necessary test data and create a CSV file that would be loaded into MongoDB using the Mongo Import Utility.<\/p>\n<div><a class=\"btn-filled btn\" href=\"https:\/\/www.iri.com\/blog\/vldb-operations\/creating-test-data-for-mongodb\/\" title=\"Creating Test Data for MongoDB\">Read More<\/a><\/div>\n","protected":false},"author":61,"featured_media":6821,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_exactmetrics_skip_tracking":false,"_exactmetrics_sitenote_active":false,"_exactmetrics_sitenote_note":"","_exactmetrics_sitenote_category":0,"footnotes":""},"categories":[108,8,29,3],"tags":[387,523,512,526,1104,1359,1361,105,955,1401,533,1273,1314,536,534,88],"class_list":["post-6791","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-big-data-2","category-data-protection","category-test-data","category-vldb-operations","tag-big-test-data","tag-build-test-data","tag-csv-file","tag-iri-rowgen","tag-json","tag-json-data","tag-json-etl","tag-loading-big-data","tag-mongo-collection-data","tag-mongo-test-data","tag-mongodb","tag-mongodb-data-masking","tag-mongodb-masking","tag-nosql-database","tag-performance-testing","tag-test-data-2"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v23.4 (Yoast SEO v23.4) - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Creating Test Data for MongoDB - IRI<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.iri.com\/blog\/vldb-operations\/creating-test-data-for-mongodb\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Creating Test Data for MongoDB\" \/>\n<meta property=\"og:description\" content=\"Introduction: This example demonstrates an older method of using IRI RowGen to generate and populate large or complex collection prototypes for testing or system capacity using flat files. As you will read, RowGen would create the necessary test data and create a CSV file that would be loaded into MongoDB using the Mongo Import Utility.Read More\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.iri.com\/blog\/vldb-operations\/creating-test-data-for-mongodb\/\" \/>\n<meta property=\"og:site_name\" content=\"IRI\" \/>\n<meta property=\"article:published_time\" content=\"2015-03-06T21:54:54+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2019-04-01T13:56:02+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2015\/02\/InfoInMongo2-e1425679035737.png\" \/>\n\t<meta property=\"og:image:width\" content=\"600\" \/>\n\t<meta property=\"og:image:height\" content=\"491\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Nathan Dymora\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Nathan Dymora\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"8 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.iri.com\/blog\/vldb-operations\/creating-test-data-for-mongodb\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.iri.com\/blog\/vldb-operations\/creating-test-data-for-mongodb\/\"},\"author\":{\"name\":\"Nathan Dymora\",\"@id\":\"https:\/\/www.iri.com\/blog\/#\/schema\/person\/6c3bde00b144e9786b3d024c1e45defa\"},\"headline\":\"Creating Test Data for MongoDB\",\"datePublished\":\"2015-03-06T21:54:54+00:00\",\"dateModified\":\"2019-04-01T13:56:02+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.iri.com\/blog\/vldb-operations\/creating-test-data-for-mongodb\/\"},\"wordCount\":1542,\"commentCount\":1,\"publisher\":{\"@id\":\"https:\/\/www.iri.com\/blog\/#organization\"},\"image\":{\"@id\":\"https:\/\/www.iri.com\/blog\/vldb-operations\/creating-test-data-for-mongodb\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2015\/02\/InfoInMongo2-e1425679035737.png\",\"keywords\":[\"big test data\",\"build test data\",\"csv file\",\"IRI RowGen\",\"JSON\",\"json data\",\"json etl\",\"loading big data\",\"Mongo collection data\",\"mongo test data\",\"MongoDB\",\"MongoDB data masking\",\"mongodb masking\",\"NoSQL database\",\"performance testing\",\"test data\"],\"articleSection\":[\"Big Data\",\"Data Masking\/Protection\",\"Test Data\",\"VLDB\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/www.iri.com\/blog\/vldb-operations\/creating-test-data-for-mongodb\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.iri.com\/blog\/vldb-operations\/creating-test-data-for-mongodb\/\",\"url\":\"https:\/\/www.iri.com\/blog\/vldb-operations\/creating-test-data-for-mongodb\/\",\"name\":\"Creating Test Data for MongoDB - IRI\",\"isPartOf\":{\"@id\":\"https:\/\/www.iri.com\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.iri.com\/blog\/vldb-operations\/creating-test-data-for-mongodb\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.iri.com\/blog\/vldb-operations\/creating-test-data-for-mongodb\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2015\/02\/InfoInMongo2-e1425679035737.png\",\"datePublished\":\"2015-03-06T21:54:54+00:00\",\"dateModified\":\"2019-04-01T13:56:02+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/www.iri.com\/blog\/vldb-operations\/creating-test-data-for-mongodb\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.iri.com\/blog\/vldb-operations\/creating-test-data-for-mongodb\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.iri.com\/blog\/vldb-operations\/creating-test-data-for-mongodb\/#primaryimage\",\"url\":\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2015\/02\/InfoInMongo2-e1425679035737.png\",\"contentUrl\":\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2015\/02\/InfoInMongo2-e1425679035737.png\",\"width\":600,\"height\":491},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.iri.com\/blog\/vldb-operations\/creating-test-data-for-mongodb\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.iri.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Creating Test Data for MongoDB\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.iri.com\/blog\/#website\",\"url\":\"https:\/\/www.iri.com\/blog\/\",\"name\":\"IRI\",\"description\":\"Total Data Management Blog\",\"publisher\":{\"@id\":\"https:\/\/www.iri.com\/blog\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.iri.com\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.iri.com\/blog\/#organization\",\"name\":\"IRI\",\"url\":\"https:\/\/www.iri.com\/blog\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.iri.com\/blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2019\/02\/iri-logo-total-data-management-small-1.png\",\"contentUrl\":\"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2019\/02\/iri-logo-total-data-management-small-1.png\",\"width\":750,\"height\":206,\"caption\":\"IRI\"},\"image\":{\"@id\":\"https:\/\/www.iri.com\/blog\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.iri.com\/blog\/#\/schema\/person\/6c3bde00b144e9786b3d024c1e45defa\",\"name\":\"Nathan Dymora\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.iri.com\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/fe3589b371c7912ed817bd9e5e443745?s=96&d=blank&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/fe3589b371c7912ed817bd9e5e443745?s=96&d=blank&r=g\",\"caption\":\"Nathan Dymora\"},\"url\":\"https:\/\/www.iri.com\/blog\/author\/nathand\/\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Creating Test Data for MongoDB - IRI","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.iri.com\/blog\/vldb-operations\/creating-test-data-for-mongodb\/","og_locale":"en_US","og_type":"article","og_title":"Creating Test Data for MongoDB","og_description":"Introduction: This example demonstrates an older method of using IRI RowGen to generate and populate large or complex collection prototypes for testing or system capacity using flat files. As you will read, RowGen would create the necessary test data and create a CSV file that would be loaded into MongoDB using the Mongo Import Utility.Read More","og_url":"https:\/\/www.iri.com\/blog\/vldb-operations\/creating-test-data-for-mongodb\/","og_site_name":"IRI","article_published_time":"2015-03-06T21:54:54+00:00","article_modified_time":"2019-04-01T13:56:02+00:00","og_image":[{"width":600,"height":491,"url":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2015\/02\/InfoInMongo2-e1425679035737.png","type":"image\/png"}],"author":"Nathan Dymora","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Nathan Dymora","Est. reading time":"8 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.iri.com\/blog\/vldb-operations\/creating-test-data-for-mongodb\/#article","isPartOf":{"@id":"https:\/\/www.iri.com\/blog\/vldb-operations\/creating-test-data-for-mongodb\/"},"author":{"name":"Nathan Dymora","@id":"https:\/\/www.iri.com\/blog\/#\/schema\/person\/6c3bde00b144e9786b3d024c1e45defa"},"headline":"Creating Test Data for MongoDB","datePublished":"2015-03-06T21:54:54+00:00","dateModified":"2019-04-01T13:56:02+00:00","mainEntityOfPage":{"@id":"https:\/\/www.iri.com\/blog\/vldb-operations\/creating-test-data-for-mongodb\/"},"wordCount":1542,"commentCount":1,"publisher":{"@id":"https:\/\/www.iri.com\/blog\/#organization"},"image":{"@id":"https:\/\/www.iri.com\/blog\/vldb-operations\/creating-test-data-for-mongodb\/#primaryimage"},"thumbnailUrl":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2015\/02\/InfoInMongo2-e1425679035737.png","keywords":["big test data","build test data","csv file","IRI RowGen","JSON","json data","json etl","loading big data","Mongo collection data","mongo test data","MongoDB","MongoDB data masking","mongodb masking","NoSQL database","performance testing","test data"],"articleSection":["Big Data","Data Masking\/Protection","Test Data","VLDB"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.iri.com\/blog\/vldb-operations\/creating-test-data-for-mongodb\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.iri.com\/blog\/vldb-operations\/creating-test-data-for-mongodb\/","url":"https:\/\/www.iri.com\/blog\/vldb-operations\/creating-test-data-for-mongodb\/","name":"Creating Test Data for MongoDB - IRI","isPartOf":{"@id":"https:\/\/www.iri.com\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.iri.com\/blog\/vldb-operations\/creating-test-data-for-mongodb\/#primaryimage"},"image":{"@id":"https:\/\/www.iri.com\/blog\/vldb-operations\/creating-test-data-for-mongodb\/#primaryimage"},"thumbnailUrl":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2015\/02\/InfoInMongo2-e1425679035737.png","datePublished":"2015-03-06T21:54:54+00:00","dateModified":"2019-04-01T13:56:02+00:00","breadcrumb":{"@id":"https:\/\/www.iri.com\/blog\/vldb-operations\/creating-test-data-for-mongodb\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.iri.com\/blog\/vldb-operations\/creating-test-data-for-mongodb\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.iri.com\/blog\/vldb-operations\/creating-test-data-for-mongodb\/#primaryimage","url":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2015\/02\/InfoInMongo2-e1425679035737.png","contentUrl":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2015\/02\/InfoInMongo2-e1425679035737.png","width":600,"height":491},{"@type":"BreadcrumbList","@id":"https:\/\/www.iri.com\/blog\/vldb-operations\/creating-test-data-for-mongodb\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.iri.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Creating Test Data for MongoDB"}]},{"@type":"WebSite","@id":"https:\/\/www.iri.com\/blog\/#website","url":"https:\/\/www.iri.com\/blog\/","name":"IRI","description":"Total Data Management Blog","publisher":{"@id":"https:\/\/www.iri.com\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.iri.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.iri.com\/blog\/#organization","name":"IRI","url":"https:\/\/www.iri.com\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.iri.com\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2019\/02\/iri-logo-total-data-management-small-1.png","contentUrl":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2019\/02\/iri-logo-total-data-management-small-1.png","width":750,"height":206,"caption":"IRI"},"image":{"@id":"https:\/\/www.iri.com\/blog\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/www.iri.com\/blog\/#\/schema\/person\/6c3bde00b144e9786b3d024c1e45defa","name":"Nathan Dymora","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.iri.com\/blog\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/fe3589b371c7912ed817bd9e5e443745?s=96&d=blank&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/fe3589b371c7912ed817bd9e5e443745?s=96&d=blank&r=g","caption":"Nathan Dymora"},"url":"https:\/\/www.iri.com\/blog\/author\/nathand\/"}]}},"jetpack_featured_media_url":"https:\/\/www.iri.com\/blog\/wp-content\/uploads\/2015\/02\/InfoInMongo2-e1425679035737.png","_links":{"self":[{"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/posts\/6791"}],"collection":[{"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/users\/61"}],"replies":[{"embeddable":true,"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/comments?post=6791"}],"version-history":[{"count":43,"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/posts\/6791\/revisions"}],"predecessor-version":[{"id":12724,"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/posts\/6791\/revisions\/12724"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/media\/6821"}],"wp:attachment":[{"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/media?parent=6791"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/categories?post=6791"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.iri.com\/blog\/wp-json\/wp\/v2\/tags?post=6791"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}