Archives
/
dhtcrawler2
mirror of https://github.com/btdigg-org/dhtcrawler2
2
0
Fork 0
Code Issues Projects Releases Wiki Activity

Commit Graph

  • 75841f8237 fix cache_indexer state not saved correctly Kevin Lynx 2013-07-16 22:48:11 +0800
  • 91ab9d9321 fix hash_reader message queue increasing memory leak bug, fix cache_indexer not sync totally Kevin Lynx 2013-07-16 22:25:42 +0800
  • c85e216951 fix cache_indexer Kevin Lynx 2013-07-16 22:24:55 +0800
  • 1ed66b3863 fix memory leak for hash_reader (message queue keep increasing), set http search result to 50 Kevin Lynx 2013-07-16 21:44:16 +0800
  • eb4f6fa9ea try to fix high cpu usage when no hash and no wait_download Kevin Lynx 2013-07-15 23:02:41 +0800
  • ff85af0806 try to fix high cpu usage when no hash and no wait_download Kevin Lynx 2013-07-15 23:01:26 +0800
  • 91731f7839 restore `top' cache Kevin Lynx 2013-07-15 22:15:06 +0800
  • c5db7ae966 restore `top' cache Kevin Lynx 2013-07-15 22:14:09 +0800
  • 31a1bd04c0 to avoid there's no hash and no wait_download, the hash reader may stop working Kevin Lynx 2013-07-15 22:04:41 +0800
  • a82fa71cf4 to avoid there's no hash and no wait_download, the hash reader may stop working Kevin Lynx 2013-07-15 22:03:52 +0800
  • b5692682e2 integrate cache_index for hash_reader, default is disabled Kevin Lynx 2013-07-15 21:40:22 +0800
  • d81d6a2fd2 integrate cache_index to hash_reader, default is disabled Kevin Lynx 2013-07-15 21:27:01 +0800
  • a0e1be291d add progress displaying for cache_indexer Kevin Lynx 2013-07-15 13:45:25 +0800
  • 0f24428faa add cache_indexer progress displaying Kevin Lynx 2013-07-15 13:39:41 +0800
  • 062130cbd9 add pre-compiled rmmseg windows 64 version Kevin Lynx 2013-07-15 12:52:43 +0800
  • 5153568dc9 add cache_indexer, not integrated now, see src/cache_indexer/readme.md Kevin Lynx 2013-07-14 22:59:47 +0800
  • c78e5d2f9c add cache_indexer, not integrated now, see src/cache_indexer/readme.md Kevin Lynx 2013-07-14 22:57:57 +0800
  • 539dd9103b hash_reader now use findAndModify to read and delete hash/wait_download Kevin Lynx 2013-07-14 15:34:52 +0800
  • 0579304407 change hash_reader read hash/wait_download using findAndModify, to avoid the read/delete two operations Kevin Lynx 2013-07-14 15:33:46 +0800
  • 356a1c0a92 fix name_seger tool Kevin Lynx 2013-07-14 13:57:49 +0800
  • 552dcb9983 fix name_segger Kevin Lynx 2013-07-14 13:53:03 +0800
  • 8d71c043bb fix name_seger tool Kevin Lynx 2013-07-14 11:44:32 +0800
  • 40bdebc5b4 change name_seger tool to multi-processes Kevin Lynx 2013-07-14 11:17:15 +0800
  • 86665cb93b only build torrent name indexes Kevin Lynx 2013-07-14 10:00:38 +0800
  • a1fc6ec3c0 add text segment config for hash_reader (text_seg), the default is simple Kevin Lynx 2013-07-13 22:27:17 +0800
  • d64630950e config text segment method [none, simple, rmmseg], default is simple Kevin Lynx 2013-07-13 22:26:15 +0800
  • 753486c16a add rmmseg, with a pre-compiled win32 version Kevin Lynx 2013-07-13 21:53:03 +0800
  • 59b54380c8 minor fix on name_seger Kevin Lynx 2013-07-13 12:05:41 +0800
  • 81e184c396 remove debug info Kevin Lynx 2013-07-13 11:51:35 +0800
  • 269584c708 add rmmseg, to segment chinese texts, add a tool to convert the existing torrent file names Kevin Lynx 2013-07-13 11:45:55 +0800
  • 77893f0759 disable numid default, update rebar to support user macro Kevin Lynx 2013-07-12 10:30:39 +0800
  • 676d354515 disable numid for sphinx default Kevin Lynx 2013-07-12 10:27:23 +0800
  • 6ddb9447ac Merge branch 'master' of github.com:kevinlynx/dhtcrawler2 Kevin Lynx 2013-07-12 09:22:09 +0800
  • 84ec8fd9dc add torrent download stats in hash reader, to debug something Kevin Lynx 2013-07-11 22:42:20 +0800
  • f5965304f7 add torrent download stats for hash reader Kevin Lynx 2013-07-11 22:38:39 +0800
  • 9d8c60f0da integrate torrent downloader monitor Kevin Lynx 2013-07-11 22:08:27 +0800
  • 1320002674 integrate torrent downloader monitor, change http today_top to show the today request count, instead total request count, remove ibrowse initial config Kevin Lynx 2013-07-11 22:01:47 +0800
  • 24b8061577 chang http top query, add a new database to map date to hashes, to support query by date range Kevin Lynx 2013-07-11 20:44:58 +0800
  • 5a0b21c7b0 chang http top query, add a new database to map date to hashes, to support query by date range Kevin Lynx 2013-07-11 20:35:16 +0800
  • fe85e0acf4 add tor_download req monitor, not integrated yet Kevin Lynx 2013-07-11 17:52:19 +0800
  • cda02229ad add tor_download req monitor, not integrated yet Kevin Lynx 2013-07-11 17:50:32 +0800
  • 42b32810c6 torbuilder(importer) fix badarith bug when there're invalid name torrent files Kevin Lynx 2013-07-11 09:06:58 +0800
  • f776cbbc46 torbuilder(importer) fix badarith bug when there're invalid name torrent files Kevin Lynx 2013-07-11 09:05:54 +0800
  • ae9f2f0c8e change http startup Kevin Lynx 2013-07-09 22:42:40 +0800
  • 4adc0a7df4 change http start to pass dbpool size Kevin Lynx 2013-07-09 22:38:34 +0800
  • 46338b02c6 rename db_hash_reader to hash_reader, change inc_announce response field Kevin Lynx 2013-07-09 21:52:47 +0800
  • 9982141e7b change hash_reader shell startup script Kevin Lynx 2013-07-09 21:50:12 +0800
  • 164c0f0f21 change inc_announce response fileds to empty (only _id) Kevin Lynx 2013-07-09 21:43:26 +0800
  • b9130ec759 use safe insert in torrent builder Kevin Lynx 2013-07-09 17:01:04 +0800
  • aa7e8bb18a use safe insert in torrent importer Kevin Lynx 2013-07-09 16:58:50 +0800
  • 5d8c198357 fix hash_reader stop working bug Kevin Lynx 2013-07-09 15:08:01 +0800
  • 40dbbeb581 fix hash_reader stop working bug when there's only wait_download hash Kevin Lynx 2013-07-09 15:06:13 +0800
  • 5a1be24b89 update torrent importer Kevin Lynx 2013-07-08 22:18:16 +0800
  • 03f98c35be update torrent importer, save process state so that can launch next time Kevin Lynx 2013-07-08 22:16:26 +0800
  • 4d31535727 separate embedded css style into page.temp Kevin Lynx 2013-07-08 12:44:54 +0800
  • 0c6097f130 separate embedded css style to page.temp Kevin Lynx 2013-07-08 12:43:27 +0800
  • 00cfa07d88 add a tool to add local torrents to database Kevin Lynx 2013-07-07 19:52:20 +0800
  • db93cd73f9 add a torrent database builder tool, to append the local torrents to database Kevin Lynx 2013-07-07 19:50:58 +0800
  • 8ee2e54e3d fix hash_reader stop working bug Kevin Lynx 2013-07-06 21:28:50 +0800
  • b42a85e929 fix hash_reader stop bug (send a wrong message) Kevin Lynx 2013-07-06 21:24:44 +0800
  • 1a9f38df49 see master branch Kevin Lynx 2013-07-06 09:42:32 +0800
  • d17d78ec08 add config to control whether load torrents from database cache Kevin Lynx 2013-07-06 09:41:51 +0800
  • cedd378fa6 IMPORTANT: add number id to torrent database Kevin Lynx 2013-07-06 01:19:06 +0800
  • 8f04efed5d IMPORTANT: add number id to torrent database Kevin Lynx 2013-07-06 01:18:25 +0800
  • 2944019ca1 see master branch for logs Kevin Lynx 2013-07-06 00:18:56 +0800
  • 01f7fd1aaf stats nice print Kevin Lynx 2013-07-05 23:49:35 +0800
  • 48022521c9 change torrent path config name Kevin Lynx 2013-07-05 23:08:26 +0800
  • ad8c931c80 stats on torrent cache used Kevin Lynx 2013-07-05 22:52:15 +0800
  • dc8cab211f fix torrent download memory leak bug Kevin Lynx 2013-07-05 21:53:22 +0800
  • 7c777f64b6 add torrent downloader Kevin Lynx 2013-07-05 21:07:35 +0800
  • 5b344c4fe2 add torrent local cache, not tested now Kevin Lynx 2013-07-03 22:56:06 +0800
  • b598fb02ea add torrent cache, not tested now Kevin Lynx 2013-07-03 22:55:17 +0800
  • 7b811ddd32 add http short keyword checking Kevin Lynx 2013-07-03 22:35:12 +0800
  • 0fcf9f8dda make the cache updating process only one, and make stats to cache Kevin Lynx 2013-07-03 21:58:06 +0800
  • 0dcf0854a8 fix http cache bug Kevin Lynx 2013-07-03 20:08:33 +0800
  • 10c11df943 fix http cache bug Kevin Lynx 2013-07-03 20:02:28 +0800
  • d82839ad27 async update cache Kevin Lynx 2013-07-03 17:53:42 +0800
  • bb2cbfcc2d async update cache Kevin Lynx 2013-07-03 17:53:13 +0800
  • 4ca1885320 update http cache in a process Kevin Lynx 2013-07-03 17:46:15 +0800
  • 530d1ce8c5 update http cache in a process Kevin Lynx 2013-07-03 17:45:35 +0800
  • 720a680743 add http cache Kevin Lynx 2013-07-03 17:20:07 +0800
  • c1f99eccd5 add http search cache Kevin Lynx 2013-07-03 17:19:28 +0800
  • bdaf17fec1 update readme Kevin Lynx 2013-07-02 20:40:55 +0800
  • 494c658aec readme Kevin Lynx 2013-07-02 20:38:53 +0800
  • 603336cfd7 update readme Kevin Lynx 2013-07-02 20:37:14 +0800
  • c82f00671a update readme Kevin Lynx 2013-07-02 20:35:13 +0800
  • 950a12169d add repl-set scripts Kevin Lynx 2013-07-02 17:43:12 +0800
  • ea6df3b2df add database repl-set scripts Kevin Lynx 2013-07-02 17:42:35 +0800
  • 2d4d3f89a7 change http to read data from mongodb slave Kevin Lynx 2013-07-02 13:57:44 +0800
  • 0a04f437cf change http to read data from mongodb slave db in repla set, can still read from master Kevin Lynx 2013-07-02 13:56:44 +0800
  • 3505116585 change hash reader waiting time to 1 min Kevin Lynx 2013-07-02 12:27:53 +0800
  • c482202fb0 change hash reader wait time to 1 min Kevin Lynx 2013-07-02 12:27:19 +0800
  • 9e38df1972 add kdht ebin Kevin Lynx 2013-07-02 11:14:42 +0800
  • 2788de582d change http interface Kevin Lynx 2013-07-02 11:05:07 +0800
  • bce9e05425 change http interfaces (modify the database query), add some config Kevin Lynx 2013-07-02 11:03:01 +0800
  • d23b3293f1 first commit Kevin Lynx 2013-07-01 23:06:18 +0800
  • 4f7fe8805e first commit Kevin Lynx 2013-07-01 22:42:14 +0800