Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DRAFT] for testing : Fix 4Gb limit for large files on Git for Windows #2179

Open
wants to merge 14 commits into
base: main
Choose a base branch
from
Open

[DRAFT] for testing : Fix 4Gb limit for large files on Git for Windows #2179

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
5b60f72
zlib.c: use size_t for size
e9925248 May 15, 2019
9c2f8de
Use size_t instead of 'unsigned long' for data in memory
tboegi Apr 21, 2019
5554ca1
Add test prereq for size_t being 64bit wide
t-b Jan 31, 2019
f05d8a6
Add test for large files on windows
t-b Jan 31, 2019
fc00458
t/helper/zlib-compile-flags: read zlib flags
Mar 17, 2019
a4b2bb5
zlib.c,http_push.c: explicit cast comparisons of potential 32bit long…
Mar 11, 2019
fb1a906
zlib.c: refactor stream chuncking to fit uLong 32bit counting
Mar 12, 2019
2ea5193
zlib.c, packfile.h,config.mak.uname: deflateBound size coercion
Mar 11, 2019
4bf4bda
index-pack.c: split bad object error messages
Mar 18, 2019
21d51c0
xcrc32(): create extended crc32 for 64 bit compatibility
Mar 21, 2019
8891566
xcrc32: use xcrc32 wrapper
Mar 21, 2019
eeb79ff
builtin/index-pack.c,csum-file.c: use size_t for memsized variables
Mar 21, 2019
a8e0ee7
packfile.c: set shift limit to sizeof(size_t), not long
Apr 22, 2019
a85708c
config.[ch]: provide config_size_t function and use it
Apr 22, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions cache.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1887,4 +1887,10 @@ int print_sha1_ellipsis(void);
/* Return 1 if the file is empty or does not exists, 0 otherwise. */
int is_empty_or_missing_file(const char *filename);

/*
* Extended crc32 with 64 bit address range
* On Windows, uInt/uLong are only 32 bits.
*/
extern uLong xcrc32(uLong crc, const unsigned char *buf, size_t bytes);

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The crc32() function is declared and defined in zlib. It would be most helpful to mention this in the code comment, along with the hint that zlib.h is already included in cache.h.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, I had though that we got it from the gnu stuff (i.e. I found it's definition from the gnu ref pages, not the zlib pages).
Double check needed. It's good if you are right (probably are..)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like I was misinformed. The crc32 is part of the zlib library and I had missed that fact.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TBH I did not know either until I looked which C standard defines the crc32() function, and did not find any.

#endif /* CACHE_H */
16 changes: 16 additions & 0 deletions wrapper.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -703,3 +703,19 @@ int is_empty_or_missing_file(const char *filename)

return !st.st_size;
}

uLong xcrc32(uLong crc, const unsigned char *buf, size_t bytes)
{
size_t bytes_rem, off;
bytes_rem = bytes;
off = 0;
while (bytes_rem) {
int crc_bytes = maximum_signed_value_of_type(int);
if (crc_bytes > bytes_rem)
crc_bytes = bytes_rem;
crc = crc32(crc, buf + off, crc_bytes);
off += crc_bytes;
bytes_rem -= crc_bytes;
}
return crc;
}