Building Better Link Previews with the URL Metadata API

Link previews have become an essential part of modern web applications. They provide users with valuable context before clicking a link, improving user experience and engagement. Our URL Metadata API makes it easy to add rich link previews to your applications.

What is the URL Metadata API?

The URL Metadata API is a powerful service that extracts metadata from any URL. It handles all the complexities of fetching, parsing, and normalizing metadata from web pages, making it easy to create rich link previews, social sharing cards, or content aggregation features.

Key Features

1. Comprehensive Metadata

   • Page title and description
   • Primary image and favicon
   • Site name and type
   • Author and publication dates
   • Keywords and language
   • Theme color

2. Smart Fallbacks

   • Handles missing metadata gracefully
   • Supports multiple metadata formats (meta tags, Open Graph, etc.)
   • Provides sensible defaults

3. Performance Optimized

   • Edge computing for fast response times
   • Automatic caching
   • Timeout protection
   • Rate limiting for fair usage

Getting Started

Using the API is straightforward. Here’s a basic example:

curl "https://blstmo.com/api/url-metadata?url=https://example.com"

The API returns a JSON response with all available metadata:

{
  "title": "Example Domain",
  "description": "Example Domain Description",
  "image": "https://example.com/image.jpg",
  "favicon": "https://example.com/favicon.ico",
  "siteName": "Example",
  "url": "https://example.com",
  "type": "website",
  "author": "Example Author",
  "publishedTime": "2024-03-06T14:30:00Z",
  "modifiedTime": "2024-03-06T15:00:00Z",
  "keywords": ["example", "domain", "website"],
  "language": "en",
  "themeColor": "#000000"
}

Common Use Cases

1. Link Preview Cards

Create engaging link previews in chat applications, social media platforms, or content management systems:

function LinkPreview({ url }: { url: string }) {
  const [metadata, setMetadata] = useState<URLMetadata | null>(null);
  const [error, setError] = useState<string | null>(null);

  useEffect(() => {
    async function fetchMetadata() {
      try {
        const data = await getURLMetadata(url);
        setMetadata(data);
      } catch (err) {
        setError(err instanceof Error ? err.message : 'Failed to fetch metadata');
      }
    }
    fetchMetadata();
  }, [url]);

  if (error) return <div className="error">{error}</div>;
  if (!metadata) return <div className="loading">Loading...</div>;

  return (
    <a href={url} className="link-preview" target="_blank" rel="noopener">
      {metadata.image && (
        <img src={metadata.image} alt={metadata.title} />
      )}
      <div className="content">
        <h3>{metadata.title}</h3>
        <p>{metadata.description}</p>
        <span className="domain">{metadata.siteName}</span>
      </div>
    </a>
  );
}

2. Content Aggregation

Build content aggregation platforms or RSS readers with rich metadata:

interface Article {
  url: string;
  metadata: URLMetadata;
}

async function aggregateContent(urls: string[]): Promise<Article[]> {
  const articles = await Promise.all(
    urls.map(async (url) => {
      try {
        const metadata = await getURLMetadata(url);
        return { url, metadata };
      } catch (error) {
        console.error(`Failed to fetch metadata for ${url}:`, error);
        return null;
      }
    })
  );

  return articles.filter((article): article is Article => article !== null);
}

3. Social Sharing

Generate social sharing previews for your content:

function SocialShare({ url }: { url: string }) {
  const [metadata, setMetadata] = useState<URLMetadata | null>(null);

  useEffect(() => {
    getURLMetadata(url).then(setMetadata).catch(console.error);
  }, [url]);

  if (!metadata) return null;

  const shareText = `Check out: ${metadata.title}`;
  const twitterUrl = `https://twitter.com/intent/tweet?text=${encodeURIComponent(shareText)}&url=${encodeURIComponent(url)}`;
  const facebookUrl = `https://www.facebook.com/sharer/sharer.php?u=${encodeURIComponent(url)}`;

  return (
    <div className="social-share">
      <h4>{metadata.title}</h4>
      <p>{metadata.description}</p>
      <div className="share-buttons">
        <a href={twitterUrl} target="_blank" rel="noopener">Share on Twitter</a>
        <a href={facebookUrl} target="_blank" rel="noopener">Share on Facebook</a>
      </div>
    </div>
  );
}

Best Practices

1. Error Handling

Always implement proper error handling:

try {
  const metadata = await getURLMetadata(url);
  // Use metadata
} catch (error) {
  if (error.message.includes('Rate limit exceeded')) {
    // Handle rate limiting
    console.error('Please wait before making more requests');
  } else if (error.message.includes('Invalid URL')) {
    // Handle invalid URLs
    console.error('Please provide a valid URL');
  } else {
    // Handle other errors
    console.error('Failed to fetch metadata:', error);
  }
}

2. Caching

Implement client-side caching to improve performance:

const metadataCache = new Map<string, { data: URLMetadata, timestamp: number }>();
const CACHE_DURATION = 3600000; // 1 hour in milliseconds

async function getCachedMetadata(url: string): Promise<URLMetadata> {
  const now = Date.now();
  const cached = metadataCache.get(url);

  if (cached && now - cached.timestamp < CACHE_DURATION) {
    return cached.data;
  }

  const metadata = await getURLMetadata(url);
  metadataCache.set(url, { data: metadata, timestamp: now });
  return metadata;
}

3. Rate Limit Handling

Implement exponential backoff for rate limit handling:

async function fetchWithRetry(url: string, maxRetries = 3): Promise<URLMetadata> {
  for (let attempt = 0; attempt < maxRetries; attempt++) {
    try {
      return await getURLMetadata(url);
    } catch (error) {
      if (error.message.includes('Rate limit exceeded')) {
        const retryAfter = parseInt(error.message.match(/\d+/)[0], 10);
        const delay = Math.min(1000 * Math.pow(2, attempt), retryAfter * 1000);
        await new Promise(resolve => setTimeout(resolve, delay));
        continue;
      }
      throw error;
    }
  }
  throw new Error('Max retries exceeded');
}

Performance Considerations

1. Batch Processing

   • Group multiple URL requests together
   • Process them in parallel with rate limiting
   • Cache results for frequently accessed URLs

2. Progressive Loading

   • Show a loading state while fetching metadata
   • Load images lazily
   • Provide fallback content

3. Error Recovery

   • Handle network errors gracefully
   • Provide fallback content when metadata is unavailable
   • Retry failed requests with exponential backoff

Security Considerations

1. URL Validation

   • Always validate URLs before making requests
   • Handle malformed URLs gracefully
   • Consider implementing URL allowlists/blocklists

2. Content Security

   • Sanitize metadata before displaying
   • Use proper HTML escaping
   • Implement CSP headers

3. Rate Limiting

   • Respect API rate limits
   • Implement client-side rate limiting
   • Cache responses to reduce API calls

Future Enhancements

We’re constantly working to improve the API. Here are some features we’re considering:

1. Batch Processing

   • Support for multiple URLs in a single request
   • Parallel processing with smart rate limiting

2. Advanced Metadata

   • Schema.org data extraction
   • JSON-LD support
   • Microdata parsing

3. Content Analysis

   • Readability scores
   • Content categorization
   • Main content extraction

Get Started Today

The URL Metadata API is ready to use in your applications. Check out our API documentation for detailed information about endpoints, parameters, and response formats.

Have questions or suggestions? We’d love to hear from you! Let us know how you’re using the API and what features you’d like to see next.

Rate Limiting

The API is rate-limited to 120 requests per minute. If you exceed this limit, you will receive a 429 status code with an error message indicating that the rate limit has been exceeded.

Usage

To use the API, send a GET request to the following endpoint:

GET /api/url-metadata?url=https://example.com

Query Parameters

• url: The URL from which to extract metadata.

Response

The API returns a JSON object with the following fields:

• title: The title of the page.
• description: The description of the page.
• image: The primary image URL.
• favicon: The favicon URL.
• siteName: The name of the site.
• url: The original URL.
• type: The type of content (e.g., article, website).
• author: The author of the content.
• publishedTime: The publication time.
• modifiedTime: The last modified time.
• keywords: An array of keywords.
• language: The language of the content.
• themeColor: The theme color of the site.

Example

{
  "title": "Example Domain",
  "description": "This domain is for use in illustrative examples in documents.",
  "image": "https://example.com/image.jpg",
  "favicon": "https://example.com/favicon.ico",
  "siteName": "Example",
  "url": "https://example.com",
  "type": "website",
  "author": "John Doe",
  "publishedTime": "2023-03-06T12:00:00Z",
  "modifiedTime": "2023-03-06T12:00:00Z",
  "keywords": ["example", "domain"],
  "language": "en",
  "themeColor": "#ffffff"
}

Error Handling

If an error occurs, the API will return a JSON object with an error field describing the issue.

Best Practices

• Cache responses to reduce the number of requests.
• Handle rate limit errors gracefully by retrying after a delay.

Support

For support, please contact our team at support@blstmo.com.