SEO Using Force-Directed Diagrams

Sometimes, the only way to understand a site is to look deep into the technology stack and gaze upon its dark, dark soul. But we at Portent try to avoid contact with evil beings. Instead, we use force-directed diagrams. I got my nerd on big-time and whipped up a few examples:

1. A healthy site thriving in its native environment
2. A sad, lonely home page, stuck out in the wilderness
3. Duplicate content invading and driving out native species
4. Pagination tunnels as they whip to and fro in the tides
5. A site relaunch that nearly gave me a heart attack
6. A web site as it explodes, spawning sub-sub-sub-sub categories galore

At the end, I link to a do-it-yourself tool you can use, and make a blatant plug for a product we’ll soon offer.

A healthy site

Here’s a force-directed diagram for a healthy web site.

And, here’s an explanation:

1. Larger dots = pages with higher internal authority
2. Longer lines = many things. Line length generally shows structural relationships between pages. Our SEO developer/genius tried to explain it to me. After he saw steam coming from my forehead, he said “Don’t overthink it. It’s complicated.” I can only suggest the same.
3. The dark blue dot is the home page.
4. Lighter blue are ‘category’ pages that link from the home page to others.

A few notes:

1. These diagrams only chart 1,600 pages. We use D3.js, and my laptop starts to glow cherry-red if we try to diagram 10,000 pages. We’re working more scalable version. It’ll be done in 2019 or so.
2. The diagrams use the output of a crawler we built here at Portent.
3. In our ‘live’ version we see page URLs. We removed them to protect the innocent.

I went through our archives and found some interesting cases:

A sad, sad homepage

See how the dark blue dot (the home page) is less popular than a hub page? This site is upside-down, with an internal page getting more internal authority than the home page itself:

This usually happens because someone said “Hey, we need a blog!” and someone else said “Ewwwwwww put that content as far away as possible.” So the blog grows into a horrifying content-crap amoeba stuffed with $50 300-word blog posts.

The problem: Yep. Content-crap amoeba. Our diagnosis was spot-on.

The fix: Remove the worst of the content and start over. Clients hate hearing their baby’s uglier than a blobfish, but you have to bite the bullet and give them the bad news.

Duplicate content nightmare

I couldn’t create site architecture this pretty:

The problem: Duplicate content. Our crawler can handle about 250,000 pages without breaking a sweat, but the first layer of this site killed it. Every orange dot is a duplicate page.

The cause? The duplicate content toilet bowl of death. The result? A Panda ‘penalty.’

The fix: Clean infinitely-chaining relative links.

The site recovered. Excellent.

Pagination tunnels

See these little, Alien-like whip-thingies?

The problem: Blog category pagination tunnels. Not necessarily a ‘problem,’ but it requires a look.

The fix: rel=next and rel=prev. I’m not a fan of nofollowing pagination. It’s part of the user experience, and you should’t hide UX from search engines. Instead, clarify pagination and ensure the tunnel has an end. That’s what rel=next and rel=prev are for.

Site relaunch. Phew

I use force-directed diagrams to compare pre- and post-relaunch sites. This is how one site looked before relaunch:

Not bad. But this is how it looked after launch:

Eaarrrggghhhh what the hell?! Something changed, and that might not be a good thing.

The problem: Actually, there is no problem. The new site separated product options and connected related content. And it removed some major duplication issues.

No fix necessary. The new site worked.

The exploding website

It took a while to figure this one out:

No duplication. No press release vomit. But the site detonates, throwing pages in all directions.

The problem: Categories. Sub-categories. Sub-sub-categories. Sub-sub-sub-categories. And so on.

The fix: A better site architecture. When the client rebuilds their site, they can restructure accordingly.

Fun pictures or serious business?

Force-directed site diagrams bring the ‘ooohhhh-aaaaahhhh’ to SEO. They aren’t essential to SEO diagnostics. But we can only do our jobs if clients understand what we’re saying. Shiny thing or not, if a tool explains an otherwise abstract concept and its importance, I use it.. I’ve ranted about this before.

The tools

If you want to create these visualizations on your own, check out Gephi. Export a site’s link graph, import it into Gephi, and generate your diagram. No, it’s not that easy. No, I can’t give free tutorials, sorry. Take a look at this page for docs and tutorials, instead.