{"topic":{"id":15091,"title":"内容推荐试验","slug":"topic","userId":4668,"categoryId":2,"createdAt":"2025-09-03 10:30:29.412348","bumpedAt":"2025-09-04 00:23:52.910377","views":265,"postsCount":3,"highestPostNumber":4,"likeCount":11,"closed":false,"pinned":false,"pinnedGlobally":false,"pinnedAt":null,"pinnedUntil":null,"isBanner":false,"excerpt":"你站试验在首页 For You Filter 中展示根据你的阅读历史推荐的话题 Sep 3, 2025, 6:31:11 PM (Asia/Singapore) 目前采用的方法： Generate cache for all users manually using rake 'recom:users' show this 'For You'-alike filter at /hot filter, shadowing existing…","deletedAt":null,"isAnonymous":false,"isPrivate":false,"isMembersOnly":false,"category":{"id":2,"name":"网站","slug":"website","color":"986a44"},"author":{"id":4668,"username":"IO","name":null,"avatarUrl":null},"realAuthor":{"id":4668,"username":"IO","name":null,"avatarUrl":null},"tags":[]},"posts":[{"id":65495,"postNumber":1,"raw":"你站试验在首页 [For You](https://xjtu.app/hot) Filter 中展示根据你的阅读历史推荐的话题\n\n[grid mode=\"tile\"]\n![image|690x47, 75%](/uploads/image/a1/24/a124268369ba993afc0e8f9f14038cbbda8f3611d1c6d09b5db87debbfdfe361.avif)\n\n![image|578x500, 50%](/uploads/image/f9/fc/f9fc44f33629f5a43dc7b35411dfd305712cb82c7d3bbd58b4f22e5f8d2f8afd.avif)\n[/grid]\n\n[date=2025-09-03 time=18:31:11 timezone=\"Asia/Singapore\"]\n目前采用的方法：\n\n\n* Generate cache for all users manually using `rake 'recom:users'`\n* show this 'For You'-alike filter at /hot filter, shadowing existing /hot\n* Recommendation method is very simple: sort viewed topics by total_msecs_viewed,\nget their related topics using embedding, andaggregate them together\n\n显然目前的方法很弱智，需要有人提出一个更好的方法。\n\nDiscourse 相关文档：\nhttps://meta.discourse.org/t/understanding-suggested-topics-logic/252936\nhttps://meta.discourse.org/t/recommendations/302355\nhttps://meta.discourse.org/t/discourse-ai-periodic-summary-reports/290236\nhttps://github.com/discourse/discourse/commit/edeb9d6cfc36ea94957efbc016ebc9ad92e46e36\nhttps://meta.discourse.org/t/could-a-list-of-recommended-topics-be-added-to-the-top-menu/289323\nhttps://blog.discourse.org/2023/12/ai-related-topics-enhancing-content-exploration/\nhttps://meta.discourse.org/t/how-does-post-tracking-work-in-discourse/115790\nhttps://meta.discourse.org/t/understanding-stats-for-topic-views-posts-read-and-reading-time/254542\nCollaborative Filtering:\nhttps://github.com/ankane/disco\nhttps://github.com/twitter/the-algorithm\nhttps://xjtu.app/t/topic/12728/15","cooked":"

你站试验在首页 For You Filter 中展示根据你的阅读历史推荐的话题

$\"image\"$

Sep 3, 2025, 6:31:11 PM (Asia/Singapore)
\n目前采用的方法：

Generate cache for all users manually using rake 'recom:users'
show this 'For You'-alike filter at /hot filter, shadowing existing /hot
Recommendation method is very simple: sort viewed topics by total_msecs_viewed,
\nget their related topics using embedding, andaggregate them together

显然目前的方法很弱智，需要有人提出一个更好的方法。

Discourse 相关文档：

\n\n\n\n\n\n\n\n\nCollaborative Filtering:\n\n\n

\n $\"pangbo\"$ \n pangbo — #15 in 关于本站公开数据提供 SQLite 下载的讨论\n

plugin_store_rows 部分插件会把用户数据存这个表里，与插件有关
\n~~post_search_data~~ search_log 会记录用户的搜索关键词记录
\npush_subscriptions 会记录用户推送的相关信息（可能包含认证信息）
\ntopic_users 包含用户的书签、话题阅读时间和进度
\npost_timings 最重量级的一个表，记录了每个用户阅读每个 post 的时间
\ntopic_link_clicks 记录了用户点击链接的记录

posts、topics、users 这几个表用白名单过滤一下列，感觉问题不大。另外有可能会导出被删除的内容，如果介意的话可以过滤一下 deleted_at。
\nlikes 是从 post_actions 里过滤出来的吗，水源没有这个表

","createdAt":"2025-09-03 10:30:29.583498","updatedAt":"2025-09-05 00:55:16.247952","replyToPostNumber":null,"replyCount":0,"likeCount":5,"wordCount":130,"liked":false,"bookmarked":false,"polls":[],"reactions":[{"reaction":"🤩️","count":2,"users":[{"id":30,"username":"一只玉米人","name":"玉米🌽"},{"id":1835,"username":"Dusk","name":"夕大画家"}],"hasReacted":false},{"reaction":"❤️","count":5,"users":[{"id":30,"username":"一只玉米人","name":"玉米🌽"},{"id":1835,"username":"Dusk","name":"夕大画家"},{"id":2902,"username":"1313","name":"ᐕ)ﾉ"},{"id":5905,"username":"PipaQinse233","name":"PipaQinse233"},{"id":7364,"username":"青年杰出贡献上交大門人","name":"紫金港第一皮革<・)))><<"}],"hasReacted":false}],"wiki":false,"version":5,"publicVersion":5,"deletedAt":null,"foldedAt":null,"isAnonymous":false,"isOwner":false,"user":{"id":4668,"username":"IO","name":null,"avatarUrl":null,"admin":true,"moderator":false,"signatureRaw":"","signatureCooked":null}},{"id":65497,"postNumber":3,"raw":"交大門搜广推项目 @tension","cooked":"

交大門搜广推项目 @tension

\n","createdAt":"2025-09-03 10:43:28.103249","updatedAt":"2025-09-03 10:43:28.103249","replyToPostNumber":null,"replyCount":0,"likeCount":2,"wordCount":9,"liked":false,"bookmarked":false,"polls":[],"reactions":[{"reaction":"❤️","count":2,"users":[{"id":1835,"username":"Dusk","name":"夕大画家"},{"id":7014,"username":"兴庆宫第一迷路幽灵","name":"post deleted by author"}],"hasReacted":false}],"wiki":false,"version":1,"publicVersion":1,"deletedAt":null,"foldedAt":null,"isAnonymous":false,"isOwner":false,"user":{"id":30,"username":"一只玉米人","name":"玉米🌽","avatarUrl":"/uploads/blob/14/96/14961aeaa46ccbb75a61eec0c27b26e74fb5d1c00ccdcbbe898ca9ea37087dd6.svg","admin":false,"moderator":true,"signatureRaw":"*年月擦身过，暂且问，会更好吗？*","signatureCooked":"

年月擦身过，暂且问，会更好吗？

\n"}},{"id":65515,"postNumber":4,"raw":"Claude:\n\nGreat idea! You can definitely build a personalized topic recommendation system on top of Discourse's existing semantic search infrastructure. Here are several approaches and Ruby libraries to achieve this:\n\n## Approach 1: User Interest Profile via Topic Embeddings\n\n**Concept:** Create a user interest vector by aggregating embeddings of topics they've viewed, then find similar topics.\n\n```ruby\n# Pseudo-code approach\nclass UserRecommendationService\n def self.recommend_topics(user, limit = 10)\n # Get user's viewed topics with recent bias\n viewed_topics = user.topic_views\n .includes(:topic)\n .where('viewed_at > ?', 30.days.ago)\n .order('viewed_at DESC')\n \n # Calculate user interest vector (weighted average of topic embeddings)\n user_vector = calculate_user_interest_vector(viewed_topics)\n \n # Find similar topics using pgvector\n similar_topics = Topic.select(\"*, embedding <-> '#{user_vector}' as distance\")\n .where.not(id: viewed_topics.pluck(:topic_id))\n .order('distance ASC')\n .limit(limit)\n \n similar_topics\n end\n \n private\n \n def self.calculate_user_interest_vector(viewed_topics)\n # Weight recent views more heavily\n weighted_embeddings = viewed_topics.map do |view|\n recency_weight = calculate_recency_weight(view.viewed_at)\n view_duration_weight = calculate_engagement_weight(view)\n \n embedding = view.topic.embedding\n weight = recency_weight * view_duration_weight\n \n embedding.map { |val| val * weight }\n end\n \n # Average the weighted embeddings\n average_vector(weighted_embeddings)\n end\nend\n```\n\n## Approach 2: Collaborative Filtering + Content-Based\n\n**Ruby Libraries:**\n- **ruby-recommender** - Simple collaborative filtering\n- **matrix** (built-in) - For similarity calculations\n- **numo-narray** - Efficient numerical arrays\n\n```ruby\n# Using ruby-recommender gem\ngem 'ruby-recommender'\n\nclass HybridRecommendationService\n def self.recommend(user_id, limit = 10)\n # Collaborative filtering component\n cf_recommendations = collaborative_filtering_recommendations(user_id)\n \n # Content-based component (using existing embeddings)\n content_recommendations = content_based_recommendations(user_id)\n \n # Combine both approaches\n hybrid_score(cf_recommendations, content_recommendations, limit)\n end\n \n private\n \n def self.collaborative_filtering_recommendations(user_id)\n # Build user-topic interaction matrix\n interactions = build_interaction_matrix\n \n recommender = Recommender::CosineSimilarity.new(interactions)\n recommender.recommendations_for(user_id)\n end\nend\n```\n\n## Approach 3: Enhanced with Machine Learning\n\n**Ruby ML Libraries:**\n- **rumale** - Comprehensive ML library\n- **ruby-opencv** - If you want to process images in topics\n- **torch-rb** - PyTorch bindings for Ruby\n\n```ruby\n# Using Rumale for clustering and recommendations\ngem 'rumale'\n\nclass MLRecommendationService\n include Rumale\n \n def self.recommend_topics(user, limit = 10)\n # Get user behavior features\n user_features = extract_user_features(user)\n \n # Use clustering to find similar users\n similar_users = find_similar_users(user_features)\n \n # Get topics liked by similar users\n recommended_topics = get_topics_from_similar_users(similar_users)\n \n # Re-rank using semantic similarity\n rerank_with_embeddings(recommended_topics, user, limit)\n end\nend\n```\n\n## Approach 4: Simple Weighted Approach (Quick Start)\n\nFor a quick implementation, you can start with this simpler approach:\n\n```ruby\nclass SimpleRecommendationService\n def self.recommend_topics(user, limit = 10)\n return Topic.latest.limit(limit) unless user.persisted?\n \n # Get user's topic categories and tags with weights\n user_interests = analyze_user_interests(user)\n \n # Score topics based on user interests\n scored_topics = Topic.published\n .where.not(id: user.topic_views.pluck(:topic_id))\n .map { |topic| [topic, calculate_interest_score(topic, user_interests)] }\n .sort_by { |_, score| -score }\n .first(limit)\n .map(&:first)\n \n scored_topics\n end\n \n private\n \n def self.analyze_user_interests(user)\n viewed_topics = user.topic_views.includes(:topic).recent\n \n # Weight by categories, tags, and recency\n interests = {\n categories: Hash.new(0),\n tags: Hash.new(0),\n users: Hash.new(0)\n }\n \n viewed_topics.each do |view|\n topic = view.topic\n recency_multiplier = recency_weight(view.viewed_at)\n \n interests[:categories][topic.category_id] += recency_multiplier\n topic.tags.each { |tag| interests[:tags][tag.name] += recency_multiplier }\n interests[:users][topic.user_id] += recency_multiplier * 0.5\n end\n \n interests\n end\nend\n```\n\n## Implementation Steps:\n\n1. **Start Simple:** Begin with the weighted approach using existing Discourse data (categories, tags, user interactions)\n\n2. **Add Vector Similarity:** Leverage your existing pgvector setup to find semantically similar topics\n\n3. **Track More Signals:** \n ```ruby\n # Add these tracking events\n - Time spent reading\n - Scroll depth\n - Likes/reactions\n - Replies/engagement\n ```\n\n4. **A/B Testing:** Use Discourse's existing A/B testing infrastructure to compare recommendation algorithms\n\n## Ruby Gems to Consider:\n\n```ruby\n# Essential gems\ngem 'matrix' # Built-in, for vector operations\ngem 'ruby-recommender' # Simple collaborative filtering\ngem 'rumale' # Machine learning toolkit\ngem 'numo-narray' # Efficient numerical computing\ngem 'redis' # For caching recommendations\n\n# Optional advanced gems\ngem 'torch-rb' # Deep learning\ngem 'lightgbm' # Gradient boosting\n```\n\n## Database Considerations:\n\nSince you already have pgvector, you can extend your schema:\n```sql\n-- Add user interest vectors\nALTER TABLE users ADD COLUMN interest_vector vector(384); -- Match your embedding dimensions\n\n-- Add interaction tracking\nCREATE TABLE user_topic_interactions (\n user_id INTEGER,\n topic_id INTEGER,\n interaction_type VARCHAR, -- 'view', 'like', 'reply', etc.\n duration INTEGER, -- time spent\n created_at TIMESTAMP\n);\n\n-- Create indexes for recommendations\nCREATE INDEX ON user_topic_interactions (user_id, created_at DESC);\n```\n\nStart with the simple weighted approach, then gradually incorporate the semantic similarity using your existing embeddings infrastructure. This will give you quick wins while building toward a more sophisticated system.","cooked":"

Claude:

Great idea! You can definitely build a personalized topic recommendation system on top of Discourse's existing semantic search infrastructure. Here are several approaches and Ruby libraries to achieve this:

Approach 1: User Interest Profile via Topic Embeddings

Concept: Create a user interest vector by aggregating embeddings of topics they've viewed, then find similar topics.

# Pseudo-code approach\nclass UserRecommendationService\n  def self.recommend_topics(user, limit = 10)\n    # Get user's viewed topics with recent bias\n    viewed_topics = user.topic_views\n                       .includes(:topic)\n                       .where('viewed_at > ?', 30.days.ago)\n                       .order('viewed_at DESC')\n    \n    # Calculate user interest vector (weighted average of topic embeddings)\n    user_vector = calculate_user_interest_vector(viewed_topics)\n    \n    # Find similar topics using pgvector\n    similar_topics = Topic.select(\"*, embedding <-> '#{user_vector}' as distance\")\n                          .where.not(id: viewed_topics.pluck(:topic_id))\n                          .order('distance ASC')\n                          .limit(limit)\n    \n    similar_topics\n  end\n  \n  private\n  \n  def self.calculate_user_interest_vector(viewed_topics)\n    # Weight recent views more heavily\n    weighted_embeddings = viewed_topics.map do |view|\n      recency_weight = calculate_recency_weight(view.viewed_at)\n      view_duration_weight = calculate_engagement_weight(view)\n      \n      embedding = view.topic.embedding\n      weight = recency_weight * view_duration_weight\n      \n      embedding.map { |val| val * weight }\n    end\n    \n    # Average the weighted embeddings\n    average_vector(weighted_embeddings)\n  end\nend\n

Approach 2: Collaborative Filtering + Content-Based

Ruby Libraries:

ruby-recommender - Simple collaborative filtering
matrix (built-in) - For similarity calculations
numo-narray - Efficient numerical arrays

# Using ruby-recommender gem\ngem 'ruby-recommender'\n\nclass HybridRecommendationService\n  def self.recommend(user_id, limit = 10)\n    # Collaborative filtering component\n    cf_recommendations = collaborative_filtering_recommendations(user_id)\n    \n    # Content-based component (using existing embeddings)\n    content_recommendations = content_based_recommendations(user_id)\n    \n    # Combine both approaches\n    hybrid_score(cf_recommendations, content_recommendations, limit)\n  end\n  \n  private\n  \n  def self.collaborative_filtering_recommendations(user_id)\n    # Build user-topic interaction matrix\n    interactions = build_interaction_matrix\n    \n    recommender = Recommender::CosineSimilarity.new(interactions)\n    recommender.recommendations_for(user_id)\n  end\nend\n

Approach 3: Enhanced with Machine Learning

Ruby ML Libraries:

rumale - Comprehensive ML library
ruby-opencv - If you want to process images in topics
torch-rb - PyTorch bindings for Ruby

# Using Rumale for clustering and recommendations\ngem 'rumale'\n\nclass MLRecommendationService\n  include Rumale\n  \n  def self.recommend_topics(user, limit = 10)\n    # Get user behavior features\n    user_features = extract_user_features(user)\n    \n    # Use clustering to find similar users\n    similar_users = find_similar_users(user_features)\n    \n    # Get topics liked by similar users\n    recommended_topics = get_topics_from_similar_users(similar_users)\n    \n    # Re-rank using semantic similarity\n    rerank_with_embeddings(recommended_topics, user, limit)\n  end\nend\n

Approach 4: Simple Weighted Approach (Quick Start)

For a quick implementation, you can start with this simpler approach:

class SimpleRecommendationService\n  def self.recommend_topics(user, limit = 10)\n    return Topic.latest.limit(limit) unless user.persisted?\n    \n    # Get user's topic categories and tags with weights\n    user_interests = analyze_user_interests(user)\n    \n    # Score topics based on user interests\n    scored_topics = Topic.published\n                         .where.not(id: user.topic_views.pluck(:topic_id))\n                         .map { |topic| [topic, calculate_interest_score(topic, user_interests)] }\n                         .sort_by { |_, score| -score }\n                         .first(limit)\n                         .map(&:first)\n    \n    scored_topics\n  end\n  \n  private\n  \n  def self.analyze_user_interests(user)\n    viewed_topics = user.topic_views.includes(:topic).recent\n    \n    # Weight by categories, tags, and recency\n    interests = {\n      categories: Hash.new(0),\n      tags: Hash.new(0),\n      users: Hash.new(0)\n    }\n    \n    viewed_topics.each do |view|\n      topic = view.topic\n      recency_multiplier = recency_weight(view.viewed_at)\n      \n      interests[:categories][topic.category_id] += recency_multiplier\n      topic.tags.each { |tag| interests[:tags][tag.name] += recency_multiplier }\n      interests[:users][topic.user_id] += recency_multiplier * 0.5\n    end\n    \n    interests\n  end\nend\n

Implementation Steps:

\n
Start Simple: Begin with the weighted approach using existing Discourse data (categories, tags, user interactions)
\n
\n
Add Vector Similarity: Leverage your existing pgvector setup to find semantically similar topics
\n

Track More Signals:

# Add these tracking events\n- Time spent reading\n- Scroll depth\n- Likes/reactions\n- Replies/engagement\n

\n
A/B Testing: Use Discourse's existing A/B testing infrastructure to compare recommendation algorithms
\n

Ruby Gems to Consider:

# Essential gems\ngem 'matrix'                    # Built-in, for vector operations\ngem 'ruby-recommender'         # Simple collaborative filtering\ngem 'rumale'                   # Machine learning toolkit\ngem 'numo-narray'             # Efficient numerical computing\ngem 'redis'                    # For caching recommendations\n\n# Optional advanced gems\ngem 'torch-rb'                # Deep learning\ngem 'lightgbm'               # Gradient boosting\n

Database Considerations:

Since you already have pgvector, you can extend your schema:

-- Add user interest vectors\nALTER TABLE users ADD COLUMN interest_vector vector(384); -- Match your embedding dimensions\n\n-- Add interaction tracking\nCREATE TABLE user_topic_interactions (\n  user_id INTEGER,\n  topic_id INTEGER,\n  interaction_type VARCHAR, -- 'view', 'like', 'reply', etc.\n  duration INTEGER,         -- time spent\n  created_at TIMESTAMP\n);\n\n-- Create indexes for recommendations\nCREATE INDEX ON user_topic_interactions (user_id, created_at DESC);\n

Start with the simple weighted approach, then gradually incorporate the semantic similarity using your existing embeddings infrastructure. This will give you quick wins while building toward a more sophisticated system.

\n","createdAt":"2025-09-04 00:23:52.910377","updatedAt":"2025-09-04 00:23:52.910377","replyToPostNumber":null,"replyCount":0,"likeCount":4,"wordCount":688,"liked":false,"bookmarked":false,"polls":[],"reactions":[{"reaction":"💯️","count":1,"users":[{"id":30,"username":"一只玉米人","name":"玉米🌽"}],"hasReacted":false},{"reaction":"❤️","count":4,"users":[{"id":30,"username":"一只玉米人","name":"玉米🌽"},{"id":1835,"username":"Dusk","name":"夕大画家"},{"id":2902,"username":"1313","name":"ᐕ)ﾉ"},{"id":7014,"username":"兴庆宫第一迷路幽灵","name":"post deleted by author"}],"hasReacted":false}],"wiki":false,"version":1,"publicVersion":1,"deletedAt":null,"foldedAt":null,"isAnonymous":false,"isOwner":false,"user":{"id":4668,"username":"IO","name":null,"avatarUrl":null,"admin":true,"moderator":false,"signatureRaw":"","signatureCooked":null}}],"hasMore":false,"lastReadPostNumber":1,"hasUrlPostNumber":false}