HackMD

Cofacts 會議記錄 - HackMD

# Cofacts 會議記錄 - [搜尋](<https://cse.google.com/cse?cx=71f4f7ee215d54fe6>)[target=_blank] ## 2024 -

arcads.ai

Arcads - Create engaging video ads using AI

Generate high-quality marketing videos quickly with Arcads, an AI-powered app that transforms a basic product link or text into engaging short video ads.

• 😱1
• 👀1

mrorz 2024-04-01 15:56:28

不過對 cofacts 來說，大概就是逐字稿處理這樣

中央社 CNA

AI及深偽影音威脅激增 專家示警台灣強化應對工具 | 政治 | 中央社 CNA

微軟研究院（Microsoft Research）兩名專家近日應英國「經濟學人」期刊邀請撰文分析人工智慧（AI）、深偽影音等新興科技對民主政治構成的挑戰，提到台灣的政府機構和研究組織缺乏及時應對工具。

• 👀3

Google

4 ways to use Search to check facts, images and sources online

For International Fact\u002DChecking Day, we’re sharing four Search features to help you evaluate information and get key context online.

台灣事實查核中心

【花蓮外海強震】地震災情影像真假一次看 查證地圖看這裡

• 😆2

mrorz 2024-04-05 19:20:30

沒聲音就會這樣～
另外應該還有 amara.org 系列
看來他華語訓練資料就是來自這些地方

HackMD

Cofacts 會議記錄 - HackMD

# Cofacts 會議記錄 - [搜尋](<https://cse.google.com/cse?cx=71f4f7ee215d54fe6>)[target=_blank] ## 2024 -

T客邦

停止讓AI再胡說八道，DeepMind 開發了「事實核查器」以糾正Claude、Gemini、GPT、PaLM-2的幻覺

解決 AI 聊天機器人幻覺問題的新方法：Google DeepMind 和史丹佛大學的 SAFE 系統

arXiv.org

Long-form factuality in large language models

Large language models (LLMs) often generate content that contains factual errors when responding to fact-seeking prompts on open-ended topics. To benchmark a model's long-form factuality in open domains, we first use GPT-4 to generate LongFact, a prompt set comprising thousands of questions spanning 38 topics. We then propose that LLM agents can be used as automated evaluators for long-form factuality through a method which we call Search-Augmented Factuality Evaluator (SAFE). SAFE utilizes an LLM to break down a long-form response into a set of individual facts and to evaluate the accuracy of each fact using a multi-step reasoning process comprising sending search queries to Google Search and determining whether a fact is supported by the search results. Furthermore, we propose extending F1 score as an aggregated metric for long-form factuality. To do so, we balance the percentage of supported facts in a response (precision) with the percentage of provided facts relative to a hyperparameter representing a user's preferred response length (recall). Empirically, we demonstrate that LLM agents can outperform crowdsourced human annotators - on a set of ~16k individual facts, SAFE agrees with crowdsourced human annotators 72% of the time, and on a random subset of 100 disagreement cases, SAFE wins 76% of the time. At the same time, SAFE is more than 20 times cheaper than human annotators. We also benchmark thirteen language models on LongFact across four model families (Gemini, GPT, Claude, and PaLM-2), finding that larger language models generally achieve better long-form factuality. LongFact, SAFE, and all experimental code are available at <https://github.com/google-deepmind/long-form-factuality>.

mrorz 2024-04-08 18:44:03

似乎是分成 supported (by world knowledge), not supported, irrelevant
讀 paper 必看的 Figure 1 在 P.1
Prompt 在 P.27

Medium

RAGFlow: Customizable, Credible, Explainable RAG engine based on document structure recognition…

Following the official open-sourcing of the AI-native database Infinity at the end of 2023, our end-to-end RAG solution, RAGFlow, was also…

gary96302000.eecs96 2024-04-08 22:54:04

Cofacts 也有服務要做到 RAG 了嗎

mrorz 2024-04-09 10:34:02

回頭看了一下
https://g0v.hackmd.io/mU8qi721RZeAQ9PDfj7XRA#AI-assisted-reply-authoring

好像確實沒有到說需要 RAG 的地步，Cofacts 資料撈出來都能塞進現在的 window size 中

gary96302000.eecs96 2024-04-09 10:39:33

Rag 其實有蠻多坑的 要採要花不少心力 不只是單純的 chunk 文章 index ,retrieve ,semantic rerank，塞進 context 的內容還要做某種程度的驗證，產出的資訊也要配 chain of thought 那類的提升準確性

每個地方都有不少梗，這是我搞了1年多的血淚經驗 😮‍💨

mrorz 2024-04-09 10:43:31

大感謝 m(_ _)m
我們如果之後要開始做上面這個 AI assisted reply authoring 的時候
再來貼到 channel 一起看看

gary96302000.eecs96 2024-04-09 10:47:02

好歐 沒問題

gary96302000.eecs96 2024-04-09 10:48:06

現在 ai 扮演的角色 是想要讓他把原文作個總結但是不評論真假嗎

mrorz 2024-04-09 11:44:05

已經做的確實是如此
接下來想加的是把 related article 加入 context 一起做總結，例如最近層出不窮的詐騙，希望他可以在讀過 related article 之後， summarize 出類似
「類似的訊息，在 2022 年、2023 年都有被傳進來過，曾被認為是詐騙」之類的回應
如果能把 citation 也補上的話最好
但 citation 看起來超不穩定的，連 OpenAI assistant API 的 annotation 也是要給不給的，可能換成在 UX 上直接揭露「我們給 AI 看了哪些 Cofacts 文章」還比較好

mrorz 2024-04-09 11:48:03

在 Prompt engineering 技巧方面，因為剛好 Cofacts 是 message + reply 的形式，我想說或許直接把 related article + reply 的 few shot example 做成像是 multi-turn conversation 試試效果

例如針對訊息 Q，我們透過 Elasticsearch 比對 article 文字內容找到文章 A1 + 回應 R1、A2+R2，以及沒有回應過的 A3
那 prompt 就可以做成下面這樣，讓 LLM 補 assistant
```User: A1
Assistant: R1 的日期、R1
User: A2
Assistant: R2 的日期、R2
User: A3
Assistant: A3 的日期，還沒有人回應 + A3 當時的 ChatGPT response
User: Q```

HackMD

Cofacts 會議記錄 - HackMD

# Cofacts 會議記錄 - [搜尋](<https://cse.google.com/cse?cx=71f4f7ee215d54fe6>)[target=_blank] ## 2024 -

Eriol Fox - opensourcedesign.​net

Hey there 👋 We (Superbloom.design) hope to submit another Design workshop + hackathon to COSCUP this year (article from 2023's workshop here) but we’d like to connect workshop activities with an OSS tool/project so that the Design workshop + hackathon can have direct outputs to an OSS.

Does anyone know any Taiwan based OSS projects that are civic tech or internet freedom related that might be interested in collaborating?

• Forwarded from #general
• 2024-04-19 17:25:41

• 🙌1

Eriol Fox - opensourcedesign.​net

Hey there 👋 We (Superbloom.design) hope to submit another Design workshop + hackathon to COSCUP this year (article from 2023's workshop here) but we’d like to connect workshop activities with an OSS tool/project so that the Design workshop + hackathon can have direct outputs to an OSS.

Does anyone know any Taiwan based OSS projects that are civic tech or internet freedom related that might be interested in collaborating?

• Forwarded from #general
• 2024-04-19 17:25:41

• 🙌1

mrorz 2024-04-22 10:18:16

你好你好～～
週一晚上八點有開發者會議唷，採線上線下 hybrid
等等會在這裡 po 會議連結～

cn.tecoffice 2024-04-22 20:57:31

謝謝~ 今天來不及參加，我再來參加之後的

HackMD

Cofacts 會議記錄 - HackMD

# Cofacts 會議記錄 - [搜尋](<https://cse.google.com/cse?cx=71f4f7ee215d54fe6>)[target=_blank] ## 2024 -