#clickhouse

2023-06-03
pm5 16:13:51
@pm5 has joined the channel
ronnywang 16:13:57
@ronnywang has joined the channel
pm5 16:14:02
他有 ngram 的樣子 https://clickhouse.com/docs/en/sql-reference/functions/splitting-merging-functions#ngrams
clickhouse.com
Functions for Splitting Strings | ClickHouse Docs
splitByChar
pm5 16:14:02
他有 ngram 的樣子 https://clickhouse.com/docs/en/sql-reference/functions/splitting-merging-functions#ngrams
pm5 16:18:36
https://clickhouse.com/docs/en/operations/utilities/clickhouse-local
pm5 16:18:36
https://clickhouse.com/docs/en/operations/utilities/clickhouse-local
clickhouse.com
clickhouse-local | ClickHouse Docs
Related Content
helloworld_bot 16:36:49
@helloworld_bot has joined the channel
pm5 16:37:23
咦
pm5 16:37:23
咦
pm5 16:39:15
好像可以?
pm5 16:39:15
好像可以?
pm5 16:44:28
這個設定方式也太神秘了。
pm5 16:44:28
這個設定方式也太神秘了。
pm5 16:48:12
...
pm5 16:48:12
...
pm5 16:50:05
...
pm5 16:50:05
...
pm5 16:52:01
.....
pm5 16:52:01
.....
pm5 16:53:32
..
pm5 16:53:32
..
pm5 16:55:15
?
pm5 16:55:15
?
pm5 16:56:11
忽然變成來做這件事了。開了一個 gateway 把 Telegram clickhouse.tw channel <--> g0v Slack clickhouse channel 連起來。
pm5 16:56:11
忽然變成來做這件事了。開了一個 gateway 把 Telegram clickhouse.tw channel <--> g0v Slack #clickhouse 連起來。
@null 16:58:30
跟 ronny 討論到一半，一時興起，沒有先跟大家說。如果有疑慮的話再麻煩跟我說喔。我們是想在 g0v 找一些喜歡玩資料的人一起來。
ronnywang 23:00:19
社會經濟資料庫的資料真的超適合 clickhouse ，我目前處理到 2013 年，資料筆數已經到六千萬筆了，處理到 2023 應該可以破億
ronnywang 23:00:19
社會經濟資料庫的資料真的超適合 clickhouse ，我目前處理到 2013 年，資料筆數已經到六千萬筆了，處理到 2023 應該可以破億
  • 👍2
tmonk 23:47:48
@felixtypingmonkey has joined the channel
2023-06-04
chihao 13:18:14
@chihao has joined the channel
chihao 13:18:35
億！
chihao 13:18:35
億！
pm5 14:36:15
set the channel description: 玩 ClickHouse 資料庫系統的頻道。與 Telegram clickhouse.tw 群組訊息同步。
ronnywang 16:10:19
giphy.webp
  • 😂2
  • 🦕1
2023-06-05
Charlie Wang 11:01:12
@cwang has joined the channel
clkao 11:06:14
@clkao has joined the channel
poga 11:23:24
@poga has joined the channel
paulpengtw 11:30:12
@paulpengtw has joined the channel
lafin 11:31:00
@lafin has joined the channel
Lucky 11:49:45
@f24931154 has joined the channel
Kyle Yu 12:22:08
@chaukwai has joined the channel
jeffery nobody-f 15:08:43
@jeffery.sac has joined the channel
A4 15:28:47
@amosli.tw has joined the channel
ronnywang 17:41:11
https://lydata.ronny-s3.click/segis.zip
社會經濟資料庫也處理好了， 2.9G ，資料筆數約2億954萬筆
ronnywang 17:41:11
https://lydata.ronny-s3.click/segis.zip
社會經濟資料庫也處理好了， 2.9G ，資料筆數約2億954萬筆
ronnywang 17:41:34
資料都是 csv
ronnywang 17:41:34
資料都是 csv
ronnywang 18:05:04
分成 2464 個 csv 檔，代表 2464 個不同個 table ，table 的名稱和裡面欄位的定義可以參考 categories.jsonl 或 category.csv
ronnywang 18:05:04
分成 2464 個 csv 檔，代表 2464 個不同個 table ，table 的名稱和裡面欄位的定義可以參考 categories.jsonl 或 category.csv
ronnywang 18:38:47
詳細欄位名稱放入 https://g0v.hackmd.io/gGGUBDEXQOKGL_feUpPekg?both
ronnywang 18:38:47
詳細欄位名稱放入 https://g0v.hackmd.io/gGGUBDEXQOKGL_feUpPekg?both
g0v.hackmd.io
適合練習 OLAP 的開放資料集 - HackMD
2023-06-06
Lynn 00:25:42
@lynn1221 has joined the channel
Carmen 00:56:30
@carmenkuo0628 has joined the channel
weiting Lin (我們的基因體時代) 12:08:31
@weitinglin66 has joined the channel
itingo 21:50:40
@itingfan has joined the channel
2023-06-08
yestin chen 10:13:43
@ggshiun has joined the channel
2023-06-09
ddio 15:36:44
@ddio has joined the channel
@null 16:03:26
請問這群組是私人群組嗎?!
@null 16:03:45
想詢問有關clickhouse的問題
@null 20:29:40
任何人都可以加入喔！
@null 20:30:07
恩恩 感謝
@null 20:30:34
有個問題想提問
helloworld_bot 20:32:24
File from [g0vbridgebot] @timdbawith comment: 我透過分佈表寫入 200000筆
但在system.query_log 顯示卻是讀取20000筆??
file 0
@null 20:59:22
沒有看到 query 本身長怎樣，有點難隔空抓藥耶。
@null 21:02:08
很單純批量insert這樣
@null 21:18:55
-- definition drop table if exists Game_Demo.demo_local on cluster cluster1 sync; CREATE TABLE Game_Demo.demo_local on cluster cluster1 ( TransferID Decimal(22, 0) , DBCreateTime DateTime64(3) ) ENGINE = MergeTree() PARTITION BY toYYYYMMDD(DBCreateTime) ORDER BY (TransferID) SETTINGS index_granularity = 8192; CREATE TABLE Game_Demo.demo ON CLUSTER cluster1 AS Game_Demo.demo_local ENGINE = Distributed('cluster1','Game_Demo','demo_local', rand()); insert into Game_Demo.demo (TransferID,DBCreateTime)VALUES (1,now()) ,(2,now()) ,(3,now()) ,(4,now()) ,(5,now()) ,(6,now()) ,(7,now()) ,(8,now()) ,(9,now()) ,(10,now()); select * from system.query_log ql where type ='QueryFinish' and query_kind ='Insert' order by event_time desc limit 10;
helloworld_bot 21:20:20
File from [g0vbridgebot] @timdba
image 2023-06-09 21-20-18
@null 21:21:24
是因為分布表收到要寫入的筆數? 所以read_rows 才會收到要批次寫入的筆數?
2023-06-16
ifengc 07:14:55
@iamifengc has joined the channel
2023-06-22
ronnywang 23:37:07
政治獻金的收支資料我補在 https://g0v.hackmd.io/gGGUBDEXQOKGL_feUpPekg
我發現用 duckdb 玩這資料還不錯方便
g0v.hackmd.io
適合練習 OLAP 的開放資料集 - HackMD
clkao 2023-06-26 19:41:36
channel 要改名了嗎 😛

順便加一下 dbt modeling
pm5 2023-06-27 08:30:08
誒 dbt model 好像不錯 XD
clkao 2023-06-27 10:00:07
this is a great project modeling many open data https://github.com/davidgasquez/datadex
clkao 2023-06-27 10:00:23
and using rill for quick local analytics
ronnywang 23:37:07
政治獻金的收支資料我補在 https://g0v.hackmd.io/gGGUBDEXQOKGL_feUpPekg
我發現用 duckdb 玩這資料還不錯方便
clkao 2023-06-26 19:41:36
channel 要改名了嗎 😛

順便加一下 dbt modeling
pm5 2023-06-27 08:30:08
誒 dbt model 好像不錯 XD
clkao 2023-06-27 10:00:07
this is a great project modeling many open data https://github.com/davidgasquez/datadex
clkao 2023-06-27 10:00:23
and using rill for quick local analytics
2023-06-26
clkao 19:41:36
Replied to a thread: 2023-06-22 23:37:07
channel 要改名了嗎 😛

順便加一下 dbt modeling
pm5 22:07:54
哇跟 telegram 的 bridge 好像斷了 XD
  • 😮1
pm5 22:07:54
哇跟 telegram 的 bridge 好像斷了 XD
2023-06-27
pm5 08:30:08
誒 dbt model 好像不錯 XD
clkao 10:00:07
this is a great project modeling many open data https://github.com/davidgasquez/datadex
clkao 10:00:23
and using rill for quick local analytics