{"id":84,"date":"2008-04-18T02:18:56","date_gmt":"2008-04-17T18:18:56","guid":{"rendered":"http:\/\/blog.urdada.net\/2008\/04\/18\/84\/"},"modified":"2008-12-22T01:25:44","modified_gmt":"2008-12-21T17:25:44","slug":"bypass-the-2gb-file-limit-on-32bit-linux","status":"publish","type":"post","link":"https:\/\/dada.tw\/blog\/2008\/04\/18\/84\/","title":{"rendered":"Bypass the 2GB file size limit on 32-bit Linux"},"content":{"rendered":"<p>Bypass the 2GB file size limit on 32-bit Linux (\u5728 Linux \u4e0a\u9762\u7a81\u7834 2GB \u7684\u6a94\u6848\u5927\u5c0f\u9650\u5236)<\/p>\n<p>\u5728 32 \u4f4d\u5143\u7684 Linux \u4e0a\u9762\u5beb\u8d85\u904e 2GB \u7684\u6a94\u6848\u6703\u767c\u751f\u932f\u8aa4\uff0c\u751a\u81f3\u5c0e\u81f4\u7a0b\u5f0f\u7d42\u6b62\u57f7\u884c<\/p>\n<p>\u9019\u662f\u56e0\u70ba Linux \u7684\u7cfb\u7d71\u5167\u90e8\u8655\u7406\u6a94\u6848\u6642\u7528\u7684\u6307\u6a19\u5b9a\u7fa9\u70ba long\uff0c\u800c long \u5728 32 \u4f4d\u5143\u7684\u7cfb\u7d71\u4e0a\u7684\u5927\u5c0f\u70ba 32 \u4f4d\u5143\uff0c\u56e0\u6b64\u6700\u5927\u53ea\u80fd\u652f\u63f4 2^31-1 = 2,147,483,647 bytes \u7b49\u65bc\u662f 2GB \u6263\u6389 1 byte \u7684\u6a94\u6848\u5927\u5c0f<\/p>\n<p>64 \u4f4d\u5143\u7684\u7cfb\u7d71 (\u4f8b\u5982 <a href=http:\/\/en.wikipedia.org\/wiki\/Amd64 target=_blank>AMD64<\/a> \u6216 <a href=http:\/\/en.wikipedia.org\/wiki\/IA64 target=_blank>IA64<\/a>) \u5247\u56e0\u70ba long \u5b9a\u7fa9\u6210 64 \u4f4d\u5143\uff0c\u6240\u4ee5\u4e0d\u6703\u6709\u554f\u984c..<\/p>\n<pre lang=\"c\">\r\n#  if __WORDSIZE == 64\r\ntypedef long int int64_t;\r\n# endif\r\n<\/pre>\n<p>\u4e0d\u904e\u5728 FreeBSD \u4e0a\u9762\uff0c\u5373\u4f7f\u662f 32 \u4f4d\u5143\u7684\u7cfb\u7d71\uff0c\u4e5f\u4e0d\u6703\u6709 2GB \u6a94\u6848\u5927\u5c0f\u7684\u9650\u5236\uff0c\u9019\u662f\u56e0\u70ba FreeBSD \u5167\u90e8\u8655\u7406\u6a94\u6848\u6642\uff0c\u672c\u4f86\u5c31\u662f\u4f7f\u7528 64 \u4f4d\u5143\u7684\u6578\u5b57\u7576\u4f5c\u6307\u6a19\uff0c\u6240\u4ee5\u4e0d\u6703\u6709\u554f\u984c<\/p>\n<p>\u56e0\u6b64\u5728 32 \u4f4d\u5143\u7684 Linux \u4e0a\u9762\uff0c\u7a0b\u5f0f\u9700\u8981\u4f5c\u4e00\u4e9b\u984d\u5916\u8655\u7406\u624d\u80fd\u6b63\u78ba\u5beb\u8d85\u904e 2GB \u7684\u6a94\u6848<\/p>\n<p>\u6211\u5011\u5148\u5beb\u4e00\u500b\u5c0f\u7a0b\u5f0f\u4f86\u6e2c\u8a66\u4e00\u4e0b (large.c)<\/p>\n<pre lang=\"c\" line=\"1\">\r\n#include <stdio.h>\r\n#include <string.h>\r\n#include <stdlib.h>\r\n#include <fcntl.h>\r\n#include <sys\/types.h>\r\n#include <sys\/stat.h>\r\n#include <signal.h>\r\n#include <unistd.h>\r\n#include <errno.h>\r\nvoid sig_xfsz(int sig)\r\n{\r\n        printf(\"ERROR: SIGXFSZ (%d) signal received!\\n\", sig);\r\n}\r\nint main()\r\n{\r\n        int     i, fd;\r\n        char    dummy[4096];\r\n\r\n        signal( SIGXFSZ, sig_xfsz );\r\n\r\n        unlink(\"large.log\");\r\n        fd = open(\"large.log\", O_CREAT|O_WRONLY, 0644 );\r\n\r\n        bzero( dummy, 4096 );\r\n        \/* 2GB = 4KB x 524288 *\/\r\n        for( i = 0 ; i < 524287 ; i++ )\r\n                write( fd, dummy, 4096 );\r\n        write( fd, dummy, 4095 );\r\n        printf(\"large.log: 2147483647 bytes\\n\");\r\n\r\n        if( write( fd, dummy, 1 ) < 0 )\r\n                printf(\"ERROR: %s [errno:%d]\\n\",strerror(errno),errno);\r\n        else\r\n                printf(\"large.log: 2147483648 bytes\\n\");\r\n\r\n        close(fd);\r\n        exit(0);\r\n}\r\n<\/pre>\n<p>\u5728 32 \u4f4d\u5143\u7684 Linux \u4e0b\u9762\uff0c\u4ee5\u4e0a\u7a0b\u5f0f\u7de8\u8b6f\u5f8c\u82e5\u6c92\u6709\u7279\u6b8a\u8655\u7406\uff0c\u57f7\u884c\u7d50\u679c\u5982\u4e0b:<\/p>\n<pre lang=\"shell\">\r\n# gcc -o large32 large.c\r\n# .\/large32\r\nlarge.log: 2147483647 bytes\r\nERROR: SIGXFSZ (25) signal received!\r\nERROR: File too large [errno:27]\r\n<\/pre>\n<p>\u5728\u5beb\u7b2c 2147483648 byte \u7684\u6642\u5019\uff0c\u7a0b\u5f0f\u6703\u6536\u5230 signal <a href=http:\/\/en.wikipedia.org\/wiki\/SIGXFSZ target=_blank>SIGXFSZ<\/a>\uff0c\u540c\u6642 write() \u6703\u56de\u50b3 -1 \u932f\u8aa4\uff0cerrno \u5247\u70ba 27 (File too large)\u3002\u66f4\u751a\u8005\uff0c\u5982\u679c\u7a0b\u5f0f\u6c92\u6709\u50cf\u4e0a\u9762\u4e00\u6a23\u53bb\u8655\u7406 SIGXFSZ \u7684\u8a71\uff0c\u5167\u5b9a\u7684 signal handler \u751a\u81f3\u6703\u9020\u6210\u7a0b\u5f0f\u505c\u6b62\u57f7\u884c\u4e26\u7522\u751f core dump<\/p>\n<p>\u63a5\u4e0b\u4f86\uff0c\u6211\u5011\u5728\u7de8\u8b6f\u540c\u4e00\u500b\u7a0b\u5f0f\u7684\u6642\u5019\u52a0\u5165 -D_FILE_OFFSET_BITS=64 \u518d\u8a66\u770b\u770b:<\/p>\n<pre lang=\"shell\">\r\n# gcc -D_FILE_OFFSET_BITS=64 -o large64 large.c\r\n# .\/large64\r\nlarge.log: 2147483647 bytes\r\nlarge.log: 2147483648 bytes\r\n<\/pre>\n<p>\u679c\u7136\u9806\u5229\u7a81\u7834 2GB \u7684\u9650\u5236\u4e86!<\/p>\n<p>\u800c\u540c\u6a23\u7684\u7a0b\u5f0f\u5728 32 \u4f4d\u5143\u7684 FreeBSD \u4e0b\u9762\uff0c\u4e0d\u8ad6\u6709\u6c92\u6709\u52a0\u9019\u500b\u5b9a\u7fa9\uff0c\u8dd1\u8d77\u4f86\u90fd\u662f\u6b63\u78ba\u7684<\/p>\n<p>\u4e0d\u904e\u8655\u7406\u9019\u4e9b\u5927\u6a94\u6848\u7684\u6642\u5019\uff0c\u9664\u4e86\u7de8\u8b6f\u7a0b\u5f0f\u6642\u7684\u53c3\u6578\u4e0d\u540c\u5916\uff0c\u6709\u4e9b\u51fd\u6578\u7684\u4f7f\u7528\u4e0a\u4e5f\u8981\u4f5c\u4e00\u4e9b\u8abf\u6574\uff0c\u4f8b\u5982 fseek() \u8207 ftell() \u9019\u5169\u500b\u539f\u672c\u4f7f\u7528\u5230 long integer \u7576\u4f5c offset \u7684\u51fd\u6578:<\/p>\n<pre lang=\"c\" line=\"1\">\r\nint fseek(FILE *stream, long offset, int whence);\r\nlong ftell(FILE *stream);\r\n<\/pre>\n<p>\u53ea\u8981\u7cfb\u7d71\u662f 32 \u4f4d\u5143\uff0c\u5373\u4f7f\u662f\u5728 FreeBSD \u4e0b\u9762\uff0c\u90fd\u9700\u8981\u6539\u70ba\u4f7f\u7528 off_t \u7684\u7248\u672c:<\/p>\n<pre lang=\"c\" line=\"1\">\r\nint fseeko(FILE *stream, off_t offset, int whence);\r\noff_t ftello(FILE *stream);\r\n<\/pre>\n<p>\u5728 Linux \u4e0b\u9762\uff0c\u5982\u679c _FILE_OFFSET_BITS \u5b9a\u7fa9\u70ba 64\uff0c\u5247 off_t \u9019\u500b\u578b\u614b\u6703\u81ea\u52d5\u8f49\u6210 64 \u4f4d\u5143\u7684\u5927\u5c0f\uff08\u5728 FreeBSD \u4e0a\u9762\uff0coff_t \u672c\u4f86\u5c31\u662f 64 \u4f4d\u5143\u7684\u5927\u5c0f)<\/p>\n<p>\u6bcf\u7a2e\u7cfb\u7d71\u652f\u63f4\u5927\u65bc 2GB \u7684\u6a94\u6848\u8b80\u5beb\u6240\u9700\u8981\u7684\u7de8\u8b6f\u9078\u9805\u90fd\u6703\u6709\u4e00\u4e9b\u5dee\u7570\uff0c\u5373\u4f7f\u662f\u540c\u6a23\u662f Linux \u4e5f\u6703\u56e0\u70ba 32 \u4f4d\u5143\u6216 64 \u4f4d\u5143\u800c\u6709\u4e0d\u540c\u3002\u6709\u4e00\u500b\u7c21\u55ae\u7684\u65b9\u6cd5\u53ef\u4ee5\u5224\u65b7\uff0c\u5c31\u662f\u5229\u7528 glibc \u63d0\u4f9b\u7684 getconf \u4f86\u53d6\u5f97\u7de8\u8b6f(compile)\u4ee5\u53ca\u9023\u7d50(linking)\u6642\u6240\u9700\u7684\u53c3\u6578:<\/p>\n<pre lang=\"shell\">\r\n# getconf LFS_CFLAGS\r\n-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64\r\n# getconf LFS_LDFLAGS \r\n\r\n#\r\n<\/pre>\n<p>\u4e0a\u9762\u662f\u5728 32 \u4f4d\u5143\u7684 Redhat Linux \u4e0a\u9762\u8dd1\u51fa\u4f86\u7684\u7d50\u679c\uff0c\u4ee3\u8868\u7684\u662f\u5728\u9019\u500b\u7cfb\u7d71\u4e0a\uff0c\u82e5\u8981\u8b93\u7a0b\u5f0f\u652f\u63f4 2GB \u7684\u6a94\u6848\u8b80\u5beb\uff0c\u7de8\u8b6f(compile)\u6642\u9700\u8981\u52a0\u4e0a -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 \u9019\u5169\u500b\u53c3\u6578\uff0c\u9023\u7d50(linking)\u6642\u5247\u4e0d\u7528\u52a0\u4efb\u4f55\u53c3\u6578<\/p>\n<p>\u53c3\u8003\u8cc7\u6599:<\/p>\n<ul>\n<li><a href=http:\/\/www.suse.de\/~aj\/linux_lfs.html target=_blank>Large File Support in Linux<\/a>\n<li><a href=http:\/\/en.wikipedia.org\/wiki\/Large_file_support target=_blank>LFS: Large File Support<\/a> (Wikipedia)\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>Bypass the 2GB file size limit on 32-bit Linux (\u5728 Linux [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[9],"tags":[],"class_list":["post-84","post","type-post","status-publish","format-standard","hentry","category-os"],"views":15672,"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/pubdi-1m","jetpack_sharing_enabled":true,"jetpack-related-posts":[{"id":85,"url":"https:\/\/dada.tw\/blog\/2008\/04\/18\/85\/","url_meta":{"origin":84,"position":0},"title":"\u4e00\u500b\u9577\u6574\u6578\u5404\u81ea\u8868\u8ff0 (in 64-bit system)","author":"dada","date":"2008-04-18","format":false,"excerpt":"Size of long integer may be different between 64-b\u2026","rel":"","context":"\u5728\u300c\u4f5c\u696d\u7cfb\u7d71\u300d\u4e2d","block_context":{"text":"\u4f5c\u696d\u7cfb\u7d71","link":"https:\/\/dada.tw\/blog\/category\/comp\/os\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":65,"url":"https:\/\/dada.tw\/blog\/2007\/11\/20\/65\/","url_meta":{"origin":84,"position":1},"title":"\u6d88\u5931\u7684\u786c\u789f\u7a7a\u9593","author":"dada","date":"2007-11-20","format":false,"excerpt":"\u8a71\u8aaa\u67d0\u4e00\u5929\uff0c\u4e00\u4f4d\u540c\u4e8b\u767c\u73fe\u67d0\u500b\u5728 UNIX \u4e0a\u7528 C \u5beb\u7684\u7a0b\u5f0f\uff0c\u8dd1\u4e00\u9663\u5b50\u5f8c\u4f3c\u4e4e\u6703\u5403\u6389\u5f88\u591a\u786c\u789f\u7a7a\u9593\uff0c\u5403\u6389\u2026","rel":"","context":"\u5728\u300c\u4f5c\u696d\u7cfb\u7d71\u300d\u4e2d","block_context":{"text":"\u4f5c\u696d\u7cfb\u7d71","link":"https:\/\/dada.tw\/blog\/category\/comp\/os\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":64,"url":"https:\/\/dada.tw\/blog\/2006\/10\/06\/64\/","url_meta":{"origin":84,"position":2},"title":"svndumpfilter with wildcards support ?","author":"dada","date":"2006-10-06","format":false,"excerpt":"Subversion \u7684 svnadmin obliterate \u529f\u80fd\u4e00\u76f4\u6c92\u6709\u5be6\u4f5c\u51fa\u4f86\uff0c\u9019\u5be6\u5728\u662f\u6709\u9ede\u2026","rel":"","context":"\u5728\u300c\u8edf\u9ad4\u300d\u4e2d","block_context":{"text":"\u8edf\u9ad4","link":"https:\/\/dada.tw\/blog\/category\/comp\/software\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":72,"url":"https:\/\/dada.tw\/blog\/2008\/01\/02\/72\/","url_meta":{"origin":84,"position":3},"title":"Change the core dump file name in Linux and FreeBSD","author":"dada","date":"2008-01-02","format":false,"excerpt":"Following the previous notes about enabling core d\u2026","rel":"","context":"\u5728\u300c\u4f5c\u696d\u7cfb\u7d71\u300d\u4e2d","block_context":{"text":"\u4f5c\u696d\u7cfb\u7d71","link":"https:\/\/dada.tw\/blog\/category\/comp\/os\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":69,"url":"https:\/\/dada.tw\/blog\/2007\/12\/24\/69\/","url_meta":{"origin":84,"position":4},"title":"Script: \u53d6\u5f97 FreeBSD CVSUP \u7684\u66f4\u65b0\u72c0\u614b","author":"dada","date":"2007-12-24","format":false,"excerpt":"\u9019\u662f\u4eca\u5e741\u6708\u5beb\u7684 script... \u7528\u4f86\u76e3\u63a7\u5404\u5927 FreeBSD CVSUP mirror \u7db2\u7ad9\u7684\u2026","rel":"","context":"\u5728\u300c\u8edf\u9ad4\u300d\u4e2d","block_context":{"text":"\u8edf\u9ad4","link":"https:\/\/dada.tw\/blog\/category\/comp\/software\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":24,"url":"https:\/\/dada.tw\/blog\/2005\/09\/08\/24\/","url_meta":{"origin":84,"position":5},"title":"URL \u542b\u4e2d\u6587\u8def\u5f91\u540d\u7a31\u7684\u7d42\u6975\u89e3\u6cd5","author":"dada","date":"2005-09-08","format":false,"excerpt":"URL \u542b\u4e2d\u6587\u8def\u5f91\u540d\u7a31\u7684\u7d42\u6975\u89e3\u6cd5 - \u5229\u7528 mod_fileiri \u89e3\u6c7a\u4e2d\u6587\u6a94\u540d\u554f\u984c \u7576\u7136\uff0c\u5c0d\u4ed8\u4e2d\u6587\u2026","rel":"","context":"\u5728\u300c\u7db2\u8def\u79d1\u6280\u300d\u4e2d","block_context":{"text":"\u7db2\u8def\u79d1\u6280","link":"https:\/\/dada.tw\/blog\/category\/network\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"amp_enabled":true,"_links":{"self":[{"href":"https:\/\/dada.tw\/blog\/wp-json\/wp\/v2\/posts\/84","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/dada.tw\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/dada.tw\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/dada.tw\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/dada.tw\/blog\/wp-json\/wp\/v2\/comments?post=84"}],"version-history":[{"count":0,"href":"https:\/\/dada.tw\/blog\/wp-json\/wp\/v2\/posts\/84\/revisions"}],"wp:attachment":[{"href":"https:\/\/dada.tw\/blog\/wp-json\/wp\/v2\/media?parent=84"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/dada.tw\/blog\/wp-json\/wp\/v2\/categories?post=84"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/dada.tw\/blog\/wp-json\/wp\/v2\/tags?post=84"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}