Five Command-Line Checks Before Publishing a Website

These examples use Bash on Linux or macOS, curl, ripgrep and Git. Replace https://example.com/article/ with a public page you own. Use a new scratch directory for response files: the curl commands overwrite the named files. Do not use authenticated pages or put credentials in the URL.

1. Search for staging addresses and unfinished copy

Run this from the source directory. Replace the staging hostname with your own, and choose markers your project uses.

rg -n -i --glob '*.html' --glob '*.php' --glob '*.md' 'staging\.example\.com|TODO-PUBLISH|lorem ipsum' .

A match is a line to inspect, not an automatic defect: a staging address may belong in documentation. Ripgrep returns 0 when it finds a match, 1 when it finds none, and 2 for an error. Do not treat every nonzero result as a successful cleanup.

This search misses ignored files by default and cannot see content held in a database. For a CMS, inspect its published response too. A clean repository search does not prove the page is clean.

2. Capture a GET response, including its failure status

curl --silent --show-error --fail --connect-timeout 10 --max-time 30 --dump-header page.headers --output page.html --write-out '%{http_code} %{content_type}\n' https://example.com/article/
printf 'curl exit: %s\n' "$?"

For an ordinary article I expect 200 and an HTML content type. The command makes a GET request, not a HEAD request. It saves headers and body separately and does not follow redirects. Run the exit-status line immediately after curl; another command would replace the status.

An HTTP error such as 404 produces a nonzero exit because of --fail. A redirect does not. Nor does a 200 response prove that the body is the article: login screens and application error pages can return 200. Open the saved response and check for the expected title and opening paragraph. Never reuse an earlier response file as evidence after a failed request.

3. Check where redirects end

For an old URL or an HTTP-to-HTTPS redirect, use a separate request:

curl --silent --show-error --fail --location --max-redirs 5 --connect-timeout 10 --max-time 30 --dump-header redirects.headers --output destination.html --write-out '%{http_code} %{url_effective} %{num_redirects}\n' http://example.com/article/
printf 'curl exit: %s\n' "$?"

Check the final URL against the destination you intended. A successful response from the homepage is still wrong if this URL should reach a specific article. The saved headers show the redirect chain. If it loops or crosses an unexpected host, fix that before changing metadata.

4. Inspect indexing instructions and the canonical together

rg -n -i 'x-robots-tag|^link:' page.headers
rg -n -i 'canonical|noindex|googlebot|name=.robots' page.html

These are search shortcuts for manual inspection, not an HTML parser. Attributes can span lines; matches can come from comments, scripts or quoted examples. Read the actual tags in the document head, and inspect every relevant response header.

For a public article intended for search, investigate a noindex directive in either a robots meta tag or an X-Robots-Tag header. Check crawler-specific directives as well. A canonical should name the preferred version of this content, not a staging host or a generic homepage. Conflicting canonicals need correction even when each URL opens.

Google must be able to crawl a page to read its indexing instructions. Blocking it in robots.txt is not a substitute for noindex. Conversely, removing noindex does not guarantee indexing. Google’s robots documentation and canonical guidance explain these separate signals.

A command-line fetch does not execute JavaScript or show crawler-specific responses. If the site changes metadata in JavaScript, or Google’s reported result differs, inspect the rendered page and the URL Inspection result rather than treating this text search as conclusive.

5. Review exactly what will ship

git status --short
git diff --check
git diff --cached --check
git diff
git diff --cached

Run these in the source checkout before release. Review both unstaged and staged changes; inspect untracked files separately. A whitespace check will not notice an incorrect redirect destination or an accidental staging hostname unless you read the patch.

For database-backed content, a clean Git tree says nothing about unpublished edits in the CMS. Preview that content, then repeat the response checks against the anonymous public URL after release. A logged-in preview can bypass caching or access controls that ordinary visitors encounter.

I would automate these checks once the expected URLs and responses are settled. First make one page pass for the right reasons, then add the known failure cases: a redirect to the wrong page, a noindex header, and an error page that returns 200. Keep a browser check for layout and navigation alongside them.

Command references: curl, ripgrep, and Git diff.