Spring RestTemplate大文件ContentLength自动更改(Spring RestTemplate Large Files ContentLength Auto Changed)

用大文件在RestTemplate上得到了一个问题。是否有最大内容长度和/或它是否会自动更改，尽管明确设置？

我有一个5GB的文件，我想通过HTTP PUT命令发送到我的CDN（Akamai）。我已经像下面那样设置了它（为了避免java堆空间错误），但是我注意到当命令执行时Content-Length被改变了。

@Autowired @Qualifier("restTemplateUpload") public void setRestTemplateUpload(RestTemplate restTemplateUpload) { this.restTemplateUpload = restTemplateUpload; SimpleClientHttpRequestFactory requestFactory = new SimpleClientHttpRequestFactory(); requestFactory.setBufferRequestBody(false); restTemplateUpload.setRequestFactory(requestFactory); }

并发送文件的调用：

Resource resource = new FileSystemResource(localFile); HttpHeaders headers = new HttpHeaders(); Map<String, String> headerValues = new HashMap<String, String>(); headerValues.put("X-Akamai...", value); //There are more headers, but you can assume this is correctly done as I've been able to transmit smaller files headerValues.put("Host", host); headers.setAll(headerValues); headers.setContentType(MediaType.MULTIPART_FORM_DATA); //I've tried using this and not using this but the result is //the same, the request sent changes the value to 1469622184 headers.setContentLength(resource.contentLength()); System.out.println(headers.getContentLength()); //Result is: 5764589480 HttpEntity<Resource> uploadRequest = new HttpEntity<Resource>(resource, headers); response = restTemplateUpload.exchange(hostPath, HttpMethod.PUT, uploadRequest, String.class);

Charles Proxy报告Content-Length 1469622184，并在请求达到该长度时连接被终止。

Java报告错误：

org.springframework.web.client.ResourceAccessException: I/O error on PUT request for "http://hidden.com/path/largefile.txt": too many bytes written;"

编辑：

更多观察，当我像这样硬编码Content-Length时：headerValues.put（“Content-Length”，“3764589480”）; 实际上不会发送Content-Length字段，而是发送标题Transfer-Encoding：chunked。

所以我运行了更多的测试，这就是发生的事情：

硬编码内容长度：查尔斯的结果

“1064589480”：发送内容长度：1064589480

“2064589480”：发送内容长度：2064589480

“3064589480”：删除内容长度，添加传输编码：分块

“3764589480”：删除Content-Length，添加Transfer-Encoding：chunked

“4294967295”：删除Content-Length，添加Transfer-Encoding：chunked

“4294967296”：发送内容长度：“0”，立即崩溃，写入的字节太多

“4764589480”：发送内容长度：“469622184”

“5764589480”：发送内容长度：“1469622184”

据我推测，在一定的数量下，RestTemplate将从Content-Length切换到Transfer-Encoding。一旦它通过4294967295，它就开始再次将Content-Length传递回0.我最好的解决方法是永远不要指定高于4294967295的值。如果它高于该阈值，那么将其简单地设置为4294967295，RestTemplate将其更改为Transfer - 编码：无论如何都要分块。是否有某种溢出/精确点问题？

Got a question on RestTemplate with Large Files. Is there a max content length and/or does it automatically get changed despite explicitly being set?

I have a file that is 5GB that I want to send to my CDN (Akamai) via HTTP PUT command. I've set it up like below (to avoid java heap space errors), however I noticed that when the command is executed the Content-Length gets changed.

And the call to send the file:

Charles Proxy reports Content-Length 1469622184 and the connection gets killed when the request reaches that length.

Java reports the error:

org.springframework.web.client.ResourceAccessException: I/O error on PUT request for "http://hidden.com/path/largefile.txt": too many bytes written;"

EDIT:

Some more observations, when I hardcode the Content-Length like so: headerValues.put("Content-Length", "3764589480"); The Content-Length field actually doesn't get sent and instead the header Transfer-Encoding: chunked gets sent.

So I ran more tests and this is what happens:

Hardcoded Content Length: Result in Charles

"1064589480": Sends Content-Length: 1064589480

"2064589480": Sends Content-Length: 2064589480

"3064589480": Removes Content-Length, Adds Transfer-Encoding: chunked

"3764589480": Removes Content-Length, Adds Transfer-Encoding: chunked

"4294967295": Removes Content-Length, Adds Transfer-Encoding: chunked

"4294967296": Sends Content-Length: "0", Immediate Crash, too many bytes written

"4764589480": Sends Content-Length: "469622184"

"5764589480": Sends Content-Length: "1469622184"

As far as I can deduce, at a certain number, RestTemplate will switch from Content-Length to Transfer-Encoding. Once it passes 4294967295, it starts passing Content-Length again back to 0. My best resolution for this is to never specify a value higher than 4294967295. If it's above that threshold, then simply set it to 4294967295 and RestTemplate will change it to Transfer-Encoding: chunked anyhow. Is there some kind of overflow/precision point issue?

最满意答案

AFAIK，4294967295字节是您可以放入“内容长度”字段中的最大数量。即使超过了这个数量，我建议将它设置为4294967295，因为RestTemplate会自动从使用Content-Length切换到使用Transfer-Encoding：分块，所以无论如何不管您放大多少。

public static final String MAX_TRANSFER_SIZE = "4294967295"; ... HttpHeaders headers = new HttpHeaders(); Map<String, String> headerValues = new HashMap<String, String>(); if (resource.contentLength() > Long.parseLong(MAX_TRANSFER_SIZE)) { // This is as high as the content length can go. At this size, // RestTemplate automatically converts from using Content-Length to // using Transfer-Encoding:chunked. headerValues.put("Content-Length", MAX_TRANSFER_SIZE); } headers.setAll(headerValues);

AFAIK, 4294967295 bytes is the maximum number you can put in the Content Length field. Even if it is over this amount, I'd recommend setting it to 4294967295 as RestTemplate will automatically switch from using Content-Length to using Transfer-Encoding: chunked so it won't matter what size you put anyhow.

更多推荐