To understand why Nexus Mods needs advertising revenue we first need to explain where we are in the grand scheme of things, what we are, and how that affects things.

Nexus Mods is an extremely large and popular resource on the internet. Currently we're ranked in the top 750 most visited websites globally, and in some countries (like the UK and Germany) we're in the top 500. Let that sink in for a second. Of all the billions of websites on the internet, Nexus Mods is in the top 750 most visited sites out there.

On top of all this traffic, we're extremely heavily focused on file sharing (the downloading of mods) and database intensive operations (highly customisable searches for said mods). Both of these site attributes are particularly expensive compared to a website that simply offers static content, with no heavy database operations or file download bandwidth to worry about. That is to say, if Nexus Mods was a wiki that simply served text and images with no file database with the same amount of traffic, our expenses would be 10-20 times less than what they are now. So a site like Nexus Mods is orders of magnitudes more expensive than more static websites out there.

So not only are we popular, but we're dealing with an extremely high volume of traffic on an extremely resource intensive style of site.

How popular is the site?

As you can see, on average, we've received 62 new pageviews and 16 new file downloads have been started every second over the course of 2016.

It's not just the total number of page visits we get per second but the actual pages being visited that are also an issue. Our most resource intensive pages, by far, are pages where a lot of customisable database calls need to be made. When I say "customisable", I mean things like filters, tag blocking, personal preferences and so on and so forth that can alter what you see compared to what someone else sees on the same page. Why? Because it's extremely difficult to cache pages that are customisable.

Caching allows us to ignore making calls to our database and instead lets us serve you "recycled" mostly static content that is orders of magnitudes faster and less resource intensive than making calls to the database.

Pages like the file search and the front page of the site are our most resource intensive pages because you can filter what you see on those pages in a number of different ways. And guess what the most traffic intensive pages on the site are? You guessed it.

In order to handle all of these database calls, we make use of a custom built database cluster. Essentially, the database cluster is 5 dedicated servers in their own rack in a datacentre hooked up to their own custom 10GbE network so they can all keep up with communicating with each other extremely fast. They make use of some highly tuned and customisable software so those 5 servers look like only 1 server to our sites, which is what a database cluster is all about.

Each of the servers boasts 192GB of RAM and 16 CPU cores. Meaning in total, our database cluster has 960GB of RAM and 80 CPU cores (and 160 threads). This database cluster is currently our largest bottleneck, meaning all those resources are not currently enough to deal with the amount of traffic we receive during times of extreme traffic. This is the reason why we had slowdowns during the Skyrim SE release. Upgrading this cluster (which would require a completely new system) would cost over $250,000, so we're now focusing on trying to get as much performance out of our code and setup as possible so we can delay that huge bill for as long as possible.

The database cluster handles all the heavy behind-the-scenes lifting, while we make use of a cloud virtual machine setup to handle serving you your pages in a timely fashion. We currently make use of 9 VMs as standard, each boasting 8 CPU cores and 32GB of RAM for a total of 72 CPU cores and 288GB of RAM. During times of heavy load we can increase the number of VMs we utilise almost indefinitely, but this obviously costs us money.

On the Nexus Mod Manager front, NMM has its own special VM that utilises 12 CPU cores and 48GB of RAM.

Sitting in front of all that is a load balancing rig of 2 further VMs with a max bandwidth limit of 500Mbit/sec (unrelated to the serving of files, that bandwidth is simply for web traffic from visiting and loading the site itself). These 2 VMs use clever (yet expensive) software to work out the best VM to send your page request to depending on how much load is on each of the VMs. Ideally, you get sent to the VM that has the least server load.

So you see, the ability for you to find exactly what you're looking for while avoiding all the stuff you don't like (be it nude mods, gore, anime, followers, etc.) comes at a serious price for us. If we removed the ability for you to block content you don't like, for example, we'd save tens of thousands of dollars in expenses each year, perhaps more, as we wouldn't need such expensive hardware.

On the download front, those 16 downloads a second come at the highest price of all. Moving to our global CDN has enabled us to remove a major bottleneck from our site. Our download system can no longer become slow or overloaded on the file-serving end during times of extreme traffic (e.g. major game launches) and allows us to gracefully, instantly, expand to accommodate larger demands. Previously we'd have a three week turnaround on implementing more file servers.

The flip side to that is that there's now no upper limit to what we will pay each month on bandwidth. Where previously our costs were capped, as we could only push as much bandwidth as our file servers would allow, any sort of "heavy" month on the CDN will result in higher costs for us. Which means we can't budget our bandwidth costs accurately any more.

]

Figure 3 - Nexus Mods bandwidth usage in 2016, showing the total number of Petabytes of data downloaded each month, the maximum bandwidth reached in Megabits per second and the 95/5 split of bandwidth.

As you can see, on average our users have downloaded 2.5 Petabytes of files each month (that's 2,500 terabytes or 2,500,000 Gigabytes in decimal) at an average 95/5 consumption of 11.5 Gbps of bandwidth.

On top of that, our file database is currently 6.5 Terabytes in size and growing daily. We not only need enough storage for all these files, but we also need to ensure we have proper redundancies and backups in place in case the worst were to happen.

Finally, for the hardware side, we also have security concerns to worry about. We spend over $30,000 a year on firewalls alone, as well as DDoS protection. Due to the size of our sites, we're often targets for extremely large botnets trying to bring the site down. Sometimes for the lulz, sometimes as a target test, other times because people just don't like us. While we can't stop the big ones, we've invested heavily into services to try and keep us up and running during most of them. Most of the time you won't even know we're under attack. And that sort of protection is expensive.

Not only do we need to pay for all this hardware, the rack space, the data centre rentals and the software that runs it all, but we also need to be able to save money for upgrades and replacement hardware. If we only make enough money to pay the current bills, we'll never be able to expand and deal with any growth in the future.

While it seems like I've gone into unnecessary detail on our extensive hardware setup, trust me when I say there's more to it (and there's more hardware and software that I won't go into!) and that we're really only scratching the surface here.

This complex system of hardware needs to be maintained, updated and kept secure. To that end (amongst many others), Nexus Mods has a dedicated team of staff.

We're now up to 10 dedicated staff members at Nexus Mods (if you include me, which I often don't), and half of those staff members have been added in the past 15 months. Why so many?

The staff at Nexus Mods are split into three distinct teams. Web programming, client programming and community:

• web programming is anything related to the sites themselves, as well as our server infrastructure
• Client programming is the Nexus Mod Manager
• Community is community management, social media, news articles, site support and PR

As you have probably gathered from the detailed hardware descriptions I've gone into above, our site and infrastructure are not only extensive and expensive, but also extremely bespoke. We not only need to ensure we can keep the sites running, but also ensure they're kept up to date with the latest versions (ensuring compatibility with our current code and setup) and handle any breakages or failures on top of actually maintaining the site code, adding new functions and features while carefully balancing our database cluster for optimum performance. It's an extremely complex and time consuming process. This is the role of the web programmers - to not only work on the site and make new features and designs for you, but to also maintain what we already have and ensure nothing breaks.

Our client programmers, the people working on the Nexus Mod Manager, are currently completely remaking the software from the ground up. Our two original NMM programmers are working under new staff member Tannin to come up with something complex, yet simple to operate.

On the community side, we're making a conscious effort to interact a lot more with the huge community we've established here over the years, trying to reconnect to it after years of being disconnected due to being too stretched on just keeping the sites up and running. We're ramping up our social media, being more active in our news section showcasing the great work and people we have in this community, rallying around and improving our moderation systems and generally trying to engage more with our community.

2016 has been a very slow year in terms of actual visible progress for you, the end user. But for us, it's been about transitioning from a small team of over-worked, tired, fedup developers just trying to keep the sites up and running into a functional, multi-purpose team that can more easily and carefully manage the myriad responsibilities and roles required to keep a site as large and popular as Nexus Mods running. The hope being that we can translate more staff into more relaxed and happy workers that will in turn be able to listen to you, the users, more about what you need that will in turn generate better features, content and working functionality for you.

All of this brings us full-circle to the original question I'm supposed to be answering. Why do we need advertising? The short version of this is; we need advertising because keeping a site like Nexus Mods up and running is extremely expensive. I hope I've explained exactly why Nexus Mods is such an expensive site to keep up and running.

But why advertising? Simply put, advertising still remains the number one way of generating income on the internet. It's also consistent. In that, typically, the more popular the site, the more page views you get, the more advertising money you receive. As a result, the hope is that the expenses relating to the site will scale with the advertising income you're generating from the site.

Case-in-point; when major game launches happen, e.g. Skyrim SE, we'll receive 25%-40% more traffic to the site over the period of a couple of months. This likely means we'll push more bandwidth, which means we'll have higher costs. However, this should be offset by the fact we've been receiving more traffic, and as a result we should be generating more income as a result. Sometimes it doesn't work out that way, for example, the eCPM (eCPM explained later) may be lower that month for some reason resulting in not making as much as you might have done in previous months, but typically, it should follow that more traffic results in more expenses but also equals more ad revenue.

People will rightly note that we also have a Premium Membership (and Supporter) system. This is absolutely correct, we do, and this service is extremely important, financially, to Nexus Mods. Having said that, Premium Membership alone would not cover the running costs of Nexus Mods and it relies a lot more on the generosity of an extremely small percentage of our users. Not only would it be unfair to rely on so few users for all the running costs of the site, but advertising also ensures that anyone and everyone, irrespective of their financial situation, can contribute to the running of the sites that they're using.

Lets not forget, that while many people bemoan the fact they have to see adverts at all and will actively find ways to remove them without helping in any way financially with the sites (e.g. by using an adblocker instead of paying $2 for a lifetime Supporter membership), many also appreciate the role they play in funding the sites, by not using an adblocker, when they can't contribute directly themselves.

It's our responsibility to ensure those adverts are relevant and secure while trying not to distract from the actual content of the site. When we fail to do that, we indirectly encourage people to explore ways to avoid seeing the ads, which is not in anyone's best interests. If everyone used an adblocker, Nexus Mods would cease to exist, after all. Currently, 45% of Nexus Mods visitors use an adblocker of some description.

I'll now go into explaining how online advertising actually works so you can better understand the process, and how bad things can happen to even the most well intentioned sites.

The vast majority of online advertising today is handled via bidder/auction systems. Each time an ad impression is generated, it is passed to a central exchange where a myriad of advertising agencies across the world will bid, in real-time, on that impression. An ad impression is one view of a single ad on a page. Nexus Mods makes use of 4 advertisements, two at the top of the page and two at the bottom. Each of those ads generates an impression, so typically 1 page view on the site generates 4 ad impressions.

To begin with, let's debunk the myth I see passed around the most from people about online advertising. If you do not click on ads on websites, ever, you will still generate revenue for that site. Let me explain why.

The vast majority of internet advertising these days is sold on a CPM basis. CPM stands for Cost-Per-Mille, and basically means "for every 1,000 ad impressions you serve, you will receive X dollars". The price of those 1,000 impressions is calculated as the eCPM (effective Cost Per Mille), so if you have an eCPM of $0.10, you'll receive $0.10 for 1,000 impressions, $1 for 10,000 impressions, $10 for 100,000 impressions, and so on and so forth. If you have an eCPM of $1 you'll receive $1 for 1,000 impressions, $10 for 10,000 impressions, $100 for 100,000 impressions, and so on and so forth. Typically, most sites will have an eCPM between $0.10 and $0.50. Sites getting more than that tend to either be in a sought after niche or have huge ad teams behind them.

As such, even if you never click on adverts on websites, the very fact it loaded on your page was enough for that site to generate ad revenue off your visit. Naturally, if you use an adblocker, that ad view is never loaded and as such, it will not generate revenue for the website.

As said before, your ad impressions get sent to a central bidding and auction system. What agencies will bid for those impressions will depend on a number of different factors. Each impression is broken down into some core information; your general location (e.g. your state and country, like "Texas, USA" or "Devon, England"), how many advertisements you've already viewed today (the more you've viewed, the more likely it is you'll have generated "banner blindness" and therefore your impression will be worth less) and what information they know about you through cookie tracking.

Cookie tracking is a contentious subject that I don't understand at its core technical level, but I'm willing to have a go at explaining what I know to you. Essentially, if you haven't done anything to mitigate cookie tracking, then ad agencies might know that you, or someone using your PC, likes video games and action movies and you're currently in the market for a new car and a foreign holiday. They gain this information through your browsing habits. The more times they get a hit on your "cookie", which follows you as you browse the internet, the more they can learn about you. Bear in mind that they do not actively store any information about you personally. So they don't know that you're "Joe Blogs living at 21 Riverside Avenue, New York", but they do know that you're anonymous user ID 12321434, who lives in New York, and likes video games and action movies.

The reason why this is extremely valuable and useful to ad agencies is because it means the ad agencies can specifically provide you with ads that are relevant to the things you like or are actively looking for. They can charge a premium to their advertising partners for this service because, as an advertiser, they'll only be advertising to people who might actually be interested in the product they're trying to sell. It's the online equivalent of Bethesda being able to choose to advertise their games in PC Gamer magazine, rather than advertising their games in the New Law Journal. They're likely to get a better return on investment advertising to gamers in PC Gamer than they are advertising to lawyers in the New Law Journal each month. That's obviously useful and valuable to them. And in turn, hopefully you'll at least see ads related to your interests and browsing habits.

The reason why it’s contentious is because some people are concerned this breaches their right to privacy. While the data is completely anonymised with no direct personal information collected, people are concerned that the data could be used to work out who people are and use the data for malicious means. I think that topic is open for debate, and while I have personal feelings on the matter (note: I do not block cookie tracking on my systems, personally) I think it's best left for another time.

It's important to note there are things you can do to mitigate cookie tracking if you don't like the idea of it or have worries about it. However, it does reduce your "worth" to the site doing the advertising and the site will likely not get as much money from your ad impressions as they would do if you had cookie tracking enabled.

Bringing it back to the bidding system, your impression will be worth various amounts to different ad agencies depending on what they are looking for. They might have a lot of high paying gaming advertisers looking for people in the US and Europe. Or they might have some beauty products in Brazil. Or cars in India. It could be anything. And how much your impression is worth will depend on millions of factors.

Each ad agency in the bidding system uses their own automated systems based around complex algorithms to work out how much your impression is worth. Once they've worked it out, they'll then submit their bid on your impression. The bidding platform will then compare all the bids from all the ad agencies and the highest value bid will "win", allowing that ad agency to show their ad to you. Typically, an impression can be worth anywhere from $0.0001 - $0.001 depending on all these different factors, with 18-35 year old males in the USA being the most valuable advertising impressions.

All that happens 4 times every time you load a page on Nexus Mods. Once per ad on our site. Thankfully, all the load is carried by the ad providers themselves and should happen asynchronously with the Nexus Mods site, which in laymans terms means the time it takes to load a page on Nexus Mods should not be affected by however long it takes for the ad provider to find the correct advertisement for you and show it to you. As such, you can have situations where you've loaded and read half the page before an ad will finally show in its proper ad place on the site.

Bringing this back to Nexus Mods, we actually use a middleman provider who handles all our advertising inventory for us. We pass all our ad impressions to them, and then they actively seek to get the best price on those impressions while taking a small cut of the proceeds.

Our middleman provider is a major player in the video game industry that has their own network of sites on which they place the same advertising that you see on Nexus Mods. This is important to us, as it means that we're "all in it together" and that they have as much vested interest as us in ensuring that the ads shown are safe and relevant to our main gamer demographic. If we report a bad ad to them, it's important that they work fast to fix it as it's highly likely that bad ad is also showing on their sites as well.

The main reason we go this route is because, if we didn't, we'd have to have our own ad operations team. That could be anywhere from 1 staff member to dozens, which is something I just don't like the idea of. With an ad ops team you can do more direct sales (e.g. circumvent the bidding system entirely and go directly to people who want to advertise their products, like game developers/publishers/hardware vendors) but the risk/reward and costs involved are tight to say the least. If you're interested about this topic in more detail then I wrote a blog post that touched on it all the way back in 2013 that is still relevant today.

The ad formats and placements on Nexus Mods should be simple and relatively unchanging.

At the top of the site is a 728x90 horizontal banner, and somewhere towards the top and on the right hand side, a 300x250 box banner. The horizontal banner is always in the same place, but the box banner changes place slightly depending on what page you're viewing at the time.

At the bottom of the site is a further 728x90 horizontal ad, and another 300x250 to the right. These ads do not change location and should always be the same no matter what page you're viewing on the site.

From time to time, the advertisements might change slightly. For example, the background banner on the site might change into an advertisement, or the 728x90 horizontal banner ad at the top of the page will become a 900x250 horizontal banner ad and take up the entire length of that content area. These are special ads that our providers have managed to directly sell, and usually command a higher price than the usual formats. They tend to be rare and it does not happen very often.

At no time should any advertisements on the site overlap any of the usual site content or prevent you from browsing or using the site as you normally would. If that does happen (e.g. content is overlapped) please inform us, as that means they aren't working properly.

These are the only ad formats Nexus Mods uses. We do not make use of any floating boxes (e.g. ads that show above the rest of the site content that "rise up" or "float down" from the sides of the page), pop-ups, pop-unders, redirects, interstitials (full page ads that show, normally with a 5-12 second timer, before you can view the site properly) or anything of a similar ilk. If you see anything like that on Nexus Mods, please report them using the procedures detailed further down this article.

Lastly, we do not wish to show any ads with auto-playing sound. That is, an ad that plays sounds or music without you interacting with the ad in any way, shape or form. Once again, if you see these, please report them using the procedures detailed further down the article.

As I hope you have sussed by now, online advertising is extremely complex and intricate. Unfortunately, it's riddled with problems as well due to poor standards in the advertising industry as a whole. It's insultingly bad and the industry is long overdue for some heavy regulation and fines when things go wrong. I would heavily advocate any legislation that put more responsibility on advertisers and agencies to clean up their act and increase and improve their vetting systems and security.

As such, we've hit this vicious cycle of ad providers showing "bad ads" which in turn spurs more users to bite the bullet and make use of ad blockers, which spurs ad providers to get more aggressive in their advertising to try and recover the losses from people using ad blockers, and so on and so forth. Ultimately, the number one thing that would help in the "war" against ad blockers is the advertising industry sorting out its act and being more responsible in their market. Of course, they rarely accept any responsibility for when things go wrong, however, so the cycle continues. It becomes a finger pointing game.

As problems can sometimes occur, a "bad ad" can be shown due to a number of different reasons. These reasons might be that the ad agency hasn't vetted the ad properly, or the ad server has been compromised by a hacker or somewhere along the advertising chain has been compromised and is serving a "bad ad".

A bad ad might be an ad that attempts to serve malware, adware or viruses, pop-ups and pop-unders on sites that don't utilise those ad formats, javascript prompts warning users they've got a virus on their system when they don't, ransomware, scripts that won't let you close your browser window, and so on and so forth. Typically, a lot of these issues are simply solved by keeping your browser and OS up to date as all the modern mainstream browsers have built in defences against such attacks. However, many people don't keep their systems up to date, which is by-and-large when most of the security issues come to a head.

I explained how the bidding system works earlier, and it's easy to understand how something might creep in if due diligence is not maintained and adhered to. With hundreds of ad agencies all bidding on impressions from one provider, that's a lot of vectors that bad people can try to attack from. It only takes one agency to slip up and not do their due-diligence for things to start going wrong. On top of that, it also makes it extremely difficult to try and work out where the bad ad is coming from and who is serving it.

Because of this, when you see a bad ad, we need as much information as possible from you so we can more easily identify where the bad ad is coming from. The bad ad may only be showing to users from a single state in the USA, or a single county in the UK, and even then might be even more specific on the demographics than that. As such, the ads I see will not be the same ads that you, or others, see. Which makes troubleshooting these things harder still.

While we have a "report this ad" link under every ad format we display on the site, there are some "bad ads" that won't be reportable via this system. For example, if a bad ad is redirecting you away from the site without your permission, or showing pop-ups or javascript prompts, the "report this ad" link will not help you or us.

Because of the aforementioned bidding system I described earlier on in this article, finding a bad ad is like finding the proverbial needle in a haystack. With potentially hundreds of ad agencies all using the same bidding platform, the bad ad could be coming from any one of those agencies. We somehow have to find that agency, report it to them, and hope they deal with it in a timely fashion. If they don't, our provider can also take that agency out of our rotation if the ad is particularly egregious.

In order to help us in our endeavours to find the bad ad and the agency responsible for displaying it as fast as possible, we need detailed information from you. And the sooner you provide us with this information, the sooner we can inform our provider who can get to work on finding and fixing the issue for all of us.

What we need:

• Let us know what anti-malware/adware/virus software you have on your PC and ensure you've run a scan that returns clean before reporting the ad to us.
• The date and time you saw this ad.
• The URL of the page you were on at the time, e.g. http://www.nexusmods.com/games/?
• Your general location - e.g. I live in Exeter, in Devon, in England. If you don't want to give the city you live in, then your state and country will suffice, e.g. Devon, England.
• A good description of what is happening that makes you think this is a bad ad. An example might be, "I was browsing the site, but immediately upon loading the page at http://www.nexusmods.com/games/? I was redirected to another website that told me I had a virus on my PC that I had to pay money to remove".
• If you get redirected or sent to another site, provide the URL that you are sent to.
• A screenshot of the bad ad really, really, REALLY helps us out as well. Please take the time to do a print-screen or use the snipping tool in windows (press the start menu, type "snipping", click "snipping tool") and upload it to your report for us to view. The snipping tool is an extremely simple and intuitive piece of software for taking pictures of what's on your screen.

We have made a forum where you can post bad ad reports that can be found under "Feedback, suggestions and questions". Ideally, please post your reports in this forum so that others can confirm that they too are seeing the same bad ad and share their own information related to it. The more reports that we receive on a specific ad, the more easy it is for us to locate and the more obvious it is that it's a serious problem, which provides clout when dealing with our provider and ad agencies. So, if someone has already reported the same bad ad that you are getting, please post in that thread to add your voice to that issue. It will help.

If for whatever reason you can't or don't want to post to that forum, you can PM me with the details.

Please remember that without these details and an accurate description about the bad ad there isn't much we can do about it.

Once we've received a good, credible report of a bad ad we pass this on to our provider who will actively work to get to the bottom of the issue for us.