Record site-name provenance instead of inferring it from the name's shape
CI / ci (push) Successful in 8m11s

Recipe extraction no longer falls back to the URL's host: a stored
recipes.site_name is now always a name the page gave (publisher or
og:site_name), and null when it named none. dinners.SourceOf is the one
documented rule for every fetched recipe's source: the page's name is
shown as it stands; otherwise the site is named by the link's host only
when the link itself is shown as free of secrets, else null. The recipe
detail, idea results and the recipe-candidate offer endpoint all apply
it; candidates no longer carry an extraction-time hostname fallback, so
their site_name is nullable (frontend updated to match).

Migration 0020 clears host-shaped site names (dotted host names, IP
addresses) stored before the change, in recipes and in legacy idea job
results, so already-stored rows read under the same rule.

Maestro 966066e3-0e25-4b8b-9bb0-0694c47c9106

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-11 10:02:44 -04:00
co-authored by Claude Opus 5.5
parent d182349646
commit 267d0be4a6
18 changed files with 360 additions and 141 deletions
+34 -37
View File
@@ -58,14 +58,14 @@ type Ingredient struct {
RawText string `json:"raw_text"`
}
// Source is the site a recipe was fetched from. SiteName is the name the
// page itself gives its site, and null when it gives none or the stored
// name could have come from a link (see pageSiteName). URL is the page's
// link only when it is verifiably free of secrets (see safeSourceURL), and
// null otherwise: a link pasted or offered as a candidate may carry a
// personal token, so a client links the credit only when URL is not null.
// Source is set for every fetched recipe, so a client always credits it:
// by SiteName, else by URL's host, else without naming the site.
// Source is the site a recipe was fetched from, as SourceOf gives it.
// SiteName is the name the page gives its site, else the host of URL when
// URL is shown, else null. URL is the page's link only when it is
// verifiably free of secrets (see safeSourceURL), and null otherwise: a
// link pasted or offered as a candidate may carry a personal token, so a
// client links the credit only when URL is not null. Source is set for
// every fetched recipe, so a client always credits it: by SiteName, else
// without naming the site.
type Source struct {
SiteName *string `json:"site_name"`
URL *string `json:"url"`
@@ -183,42 +183,39 @@ func readRecipe(ctx context.Context, q *db.Queries, recipeID string, d *Detail)
}
// SourceOf is the Source of a recipe fetched from sourceURL whose stored
// site name is siteName: the name only when the page gave it (see
// pageSiteName) and the link only when it is free of secrets (see
// safeSourceURL). It is the one way a fetched recipe's source reaches a
// client. A sourceURL that does not parse is an error, whether or not a
// site name is stored.
// site name is siteName, the name its page gave (recipes.Recipe.SiteName),
// null when the page named none. It is the one rule by which a fetched
// recipe's source reaches a client — the recipe detail, an idea and a
// recipe candidate offered for a dinner alike:
//
// - the link is shown only when it is free of secrets (see
// safeSourceURL);
// - a site name the page gave is shown as it stands, as the page
// publishes it to everyone who reads it;
// - otherwise the site is named by the link's host without a leading
// "www.", and only when the link itself is shown, so the name tells
// nothing the shown link does not; else it is null.
//
// A host is never stored as a site name, so a stored name is always the
// page's: migration 0020 cleared the names stored before that, when a
// page naming no site was stored under its host. A sourceURL that does not
// parse is an error, whether or not a site name is stored.
func SourceOf(siteName sql.NullString, sourceURL string) (Source, error) {
link, err := url.Parse(sourceURL)
if err != nil {
// Not wrapped: url.Parse's error quotes the link.
return Source{}, errors.New("source url does not parse")
}
return Source{SiteName: pageSiteName(siteName, link), URL: safeSourceURL(sourceURL)}, nil
}
// pageSiteName is the stored site name of a recipe fetched from link
// when it is one the page gave, else nil. A page that names no site is
// stored under recipes.HostSiteName of the URL it was served from: the
// pasted link's host, which may hold a personal token (a private
// subdomain), or that of a page the link redirected to, which is not
// stored. Nothing records which a stored name is, so any name that could
// be a host — the link's own host, an IP address, or a dotted host name —
// is treated as one and withheld. A site that names itself as a domain,
// such as "Food.com", loses its name with them: a credit that does not
// name the site is better than one that leaks a link.
func pageSiteName(stored sql.NullString, link *url.URL) *string {
if !stored.Valid {
return nil
src := Source{URL: safeSourceURL(sourceURL)}
switch {
case siteName.Valid && siteName.String != "":
name := siteName.String
src.SiteName = &name
case src.URL != nil:
host := strings.TrimPrefix(link.Hostname(), "www.")
src.SiteName = &host
}
name := stored.String
if strings.EqualFold(name, recipes.HostSiteName(link)) || net.ParseIP(name) != nil {
return nil
}
if u, err := url.Parse("https://" + name + "/"); err == nil && u.Host == name && strings.Contains(name, ".") {
return nil
}
return &name
return src, nil
}
// safeSourceURL is sourceURL when it is verifiably free of secrets — an