Spring ThreadPoolTaskExecutor with fixed delay?

up vote
0
down vote

favorite

I have a Job which is scheduled to run every hour which uses Spring ThreadPoolTaskExecutor bean to fire simultaneous calls (close to 100 calls every hour) to external API.

@Bean

public TaskExecutor getExecutor() {

    ThreadPoolTaskExecutor threadPoolTaskExecutor = new ThreadPoolTaskExecutor();

    threadPoolTaskExecutor.setCorePoolSize(10);

    threadPoolTaskExecutor.setMaxPoolSize(20);

    return threadPoolTaskExecutor;

}

Now, the external API has throttled the number of requests and allows one request per 30 secs. I will have to wait 30 secs before making each call.

In this case, I see use of ThreadPoolTaskExecutor is no longer helpful. Will ThreadPoolTaskScheduler with Fixed Delay configuration work?

What is the best way to handle such type of API throttling? Please help

I'm on Java 8, Spring Boot 2.0.1 if that helps

asked Nov 8 at 9:53

jusermar10

733721

You are actually aware of the solution, a ThreadPoolTaskScheduler scheduler with a 30 seconds delayed trigger would work. Why wouldn't you just implement that? Are there any reasons?
– tmarwen
Nov 8 at 10:13

@tmarwen I have used @Scheduled with a cron expression in the actual Job which runs every hour. Can I have a concurrent fixed delay schedule in ThreadPoolTaskScheduler ?
– jusermar10
Nov 8 at 10:19

@jusemar10 You might like to control the schedule on your @Scheduled annotation and leave the ThreadPoolTaskScheduler alone.
– Arun Patra
Nov 8 at 12:54

1

I assume what you actually need is some kind of way to spread out the calls so it knows how much time has passed since the last one, and puts them on the queue if tasks need throttling. Although none of this helps if you don't change your app in a meaningfull way: you have to change it so it goes from 100 calls a minute to at most 2 calls. This isn't achievable with just throttling the executor.
– M. Prokhorov
Nov 8 at 13:26

... because if you just change the executor, and still load it with 100 tasks while it can process 2 at most, the tasks queue would get polluted very fast (98 a minute at least, to be precise).
– M. Prokhorov
Nov 8 at 13:28

|
show 1 more comment

up vote
0
down vote

favorite

I have a Job which is scheduled to run every hour which uses Spring ThreadPoolTaskExecutor bean to fire simultaneous calls (close to 100 calls every hour) to external API.

@Bean

public TaskExecutor getExecutor() {

    ThreadPoolTaskExecutor threadPoolTaskExecutor = new ThreadPoolTaskExecutor();

    threadPoolTaskExecutor.setCorePoolSize(10);

    threadPoolTaskExecutor.setMaxPoolSize(20);

    return threadPoolTaskExecutor;

}

Now, the external API has throttled the number of requests and allows one request per 30 secs. I will have to wait 30 secs before making each call.

In this case, I see use of ThreadPoolTaskExecutor is no longer helpful. Will ThreadPoolTaskScheduler with Fixed Delay configuration work?

What is the best way to handle such type of API throttling? Please help

I'm on Java 8, Spring Boot 2.0.1 if that helps

asked Nov 8 at 9:53

jusermar10

733721

You are actually aware of the solution, a ThreadPoolTaskScheduler scheduler with a 30 seconds delayed trigger would work. Why wouldn't you just implement that? Are there any reasons?
– tmarwen
Nov 8 at 10:13

@tmarwen I have used @Scheduled with a cron expression in the actual Job which runs every hour. Can I have a concurrent fixed delay schedule in ThreadPoolTaskScheduler ?
– jusermar10
Nov 8 at 10:19

@jusemar10 You might like to control the schedule on your @Scheduled annotation and leave the ThreadPoolTaskScheduler alone.
– Arun Patra
Nov 8 at 12:54

1

I assume what you actually need is some kind of way to spread out the calls so it knows how much time has passed since the last one, and puts them on the queue if tasks need throttling. Although none of this helps if you don't change your app in a meaningfull way: you have to change it so it goes from 100 calls a minute to at most 2 calls. This isn't achievable with just throttling the executor.
– M. Prokhorov
Nov 8 at 13:26

... because if you just change the executor, and still load it with 100 tasks while it can process 2 at most, the tasks queue would get polluted very fast (98 a minute at least, to be precise).
– M. Prokhorov
Nov 8 at 13:28

|
show 1 more comment

up vote
0
down vote

favorite

I have a Job which is scheduled to run every hour which uses Spring ThreadPoolTaskExecutor bean to fire simultaneous calls (close to 100 calls every hour) to external API.

@Bean

public TaskExecutor getExecutor() {

    ThreadPoolTaskExecutor threadPoolTaskExecutor = new ThreadPoolTaskExecutor();

    threadPoolTaskExecutor.setCorePoolSize(10);

    threadPoolTaskExecutor.setMaxPoolSize(20);

    return threadPoolTaskExecutor;

}

Now, the external API has throttled the number of requests and allows one request per 30 secs. I will have to wait 30 secs before making each call.

In this case, I see use of ThreadPoolTaskExecutor is no longer helpful. Will ThreadPoolTaskScheduler with Fixed Delay configuration work?

What is the best way to handle such type of API throttling? Please help

I'm on Java 8, Spring Boot 2.0.1 if that helps

asked Nov 8 at 9:53

jusermar10

733721

I have a Job which is scheduled to run every hour which uses Spring ThreadPoolTaskExecutor bean to fire simultaneous calls (close to 100 calls every hour) to external API.

@Bean

public TaskExecutor getExecutor() {

    ThreadPoolTaskExecutor threadPoolTaskExecutor = new ThreadPoolTaskExecutor();

    threadPoolTaskExecutor.setCorePoolSize(10);

    threadPoolTaskExecutor.setMaxPoolSize(20);

    return threadPoolTaskExecutor;

}

Now, the external API has throttled the number of requests and allows one request per 30 secs. I will have to wait 30 secs before making each call.

In this case, I see use of ThreadPoolTaskExecutor is no longer helpful. Will ThreadPoolTaskScheduler with Fixed Delay configuration work?

What is the best way to handle such type of API throttling? Please help

I'm on Java 8, Spring Boot 2.0.1 if that helps

java spring multithreading spring-boot threadpoolexecutor

asked Nov 8 at 9:53

jusermar10

733721

asked Nov 8 at 9:53

jusermar10

733721

asked Nov 8 at 9:53

jusermar10

733721

asked Nov 8 at 9:53

jusermar10

733721

asked Nov 8 at 9:53

jusermar10

733721

You are actually aware of the solution, a ThreadPoolTaskScheduler scheduler with a 30 seconds delayed trigger would work. Why wouldn't you just implement that? Are there any reasons?
– tmarwen
Nov 8 at 10:13

@tmarwen I have used @Scheduled with a cron expression in the actual Job which runs every hour. Can I have a concurrent fixed delay schedule in ThreadPoolTaskScheduler ?
– jusermar10
Nov 8 at 10:19

@jusemar10 You might like to control the schedule on your @Scheduled annotation and leave the ThreadPoolTaskScheduler alone.
– Arun Patra
Nov 8 at 12:54

1

I assume what you actually need is some kind of way to spread out the calls so it knows how much time has passed since the last one, and puts them on the queue if tasks need throttling. Although none of this helps if you don't change your app in a meaningfull way: you have to change it so it goes from 100 calls a minute to at most 2 calls. This isn't achievable with just throttling the executor.
– M. Prokhorov
Nov 8 at 13:26

... because if you just change the executor, and still load it with 100 tasks while it can process 2 at most, the tasks queue would get polluted very fast (98 a minute at least, to be precise).
– M. Prokhorov
Nov 8 at 13:28

|
show 1 more comment

You are actually aware of the solution, a ThreadPoolTaskScheduler scheduler with a 30 seconds delayed trigger would work. Why wouldn't you just implement that? Are there any reasons?
– tmarwen
Nov 8 at 10:13

@tmarwen I have used @Scheduled with a cron expression in the actual Job which runs every hour. Can I have a concurrent fixed delay schedule in ThreadPoolTaskScheduler ?
– jusermar10
Nov 8 at 10:19

@jusemar10 You might like to control the schedule on your @Scheduled annotation and leave the ThreadPoolTaskScheduler alone.
– Arun Patra
Nov 8 at 12:54

1

I assume what you actually need is some kind of way to spread out the calls so it knows how much time has passed since the last one, and puts them on the queue if tasks need throttling. Although none of this helps if you don't change your app in a meaningfull way: you have to change it so it goes from 100 calls a minute to at most 2 calls. This isn't achievable with just throttling the executor.
– M. Prokhorov
Nov 8 at 13:26

... because if you just change the executor, and still load it with 100 tasks while it can process 2 at most, the tasks queue would get polluted very fast (98 a minute at least, to be precise).
– M. Prokhorov
Nov 8 at 13:28

You are actually aware of the solution, a ThreadPoolTaskScheduler scheduler with a 30 seconds delayed trigger would work. Why wouldn't you just implement that? Are there any reasons?
– tmarwen
Nov 8 at 10:13

@tmarwen I have used @Scheduled with a cron expression in the actual Job which runs every hour. Can I have a concurrent fixed delay schedule in ThreadPoolTaskScheduler ?
– jusermar10
Nov 8 at 10:19

@jusemar10 You might like to control the schedule on your @Scheduled annotation and leave the ThreadPoolTaskScheduler alone.
– Arun Patra
Nov 8 at 12:54

I assume what you actually need is some kind of way to spread out the calls so it knows how much time has passed since the last one, and puts them on the queue if tasks need throttling. Although none of this helps if you don't change your app in a meaningfull way: you have to change it so it goes from 100 calls a minute to at most 2 calls. This isn't achievable with just throttling the executor.
– M. Prokhorov
Nov 8 at 13:26

... because if you just change the executor, and still load it with 100 tasks while it can process 2 at most, the tasks queue would get polluted very fast (98 a minute at least, to be precise).
– M. Prokhorov
Nov 8 at 13:28

|
show 1 more comment

1 Answer
1

active

oldest

votes

up vote
0
down vote

You could do the following.

Configure the TaskExecutor. You have already done this.

Write a method that you want to invoke in a scheduled fashion and annotate with @Scheduled.
```
 @Scheduled(fixedDelay = 3600000)

 void doSomething() {

 }
```

The code above will trigger every 1 hour.

answered Nov 8 at 12:53

Arun Patra

43328

I have this setup already. My question is how to limit the API call which is currently fired about 100 times in parallel every time the job runs.
– jusermar10
Nov 8 at 13:13

This will always throttle the task, even if there were no tasks run for the past 10 minutes.
– M. Prokhorov
Nov 8 at 13:24

Agree with @M.Prokhorov . Your problem is not in the domain of the Scheduling infrastructure that Spring provides. You would probably have to build your own logic. Maybe your own custom logic (or reuse lower level of task management infrastructure from Java itself or Spring) to submit tasks to an intermediate processor. That intermediate processor would keep track of timestamps, and would coordinate a proper delay between consecutive submissions.
– Arun Patra
Nov 9 at 6:04

add a comment |

Your Answer

StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});

}
});

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53205239%2fspring-threadpooltaskexecutor-with-fixed-delay%23new-answer', 'question_page');
}
);

Post as a guest

Name

1 Answer
1

active

oldest

votes

1 Answer
1

active

oldest

votes

up vote
0
down vote

You could do the following.

Configure the TaskExecutor. You have already done this.

Write a method that you want to invoke in a scheduled fashion and annotate with @Scheduled.
```
 @Scheduled(fixedDelay = 3600000)

 void doSomething() {

 }
```

The code above will trigger every 1 hour.

answered Nov 8 at 12:53

Arun Patra

43328

I have this setup already. My question is how to limit the API call which is currently fired about 100 times in parallel every time the job runs.
– jusermar10
Nov 8 at 13:13

This will always throttle the task, even if there were no tasks run for the past 10 minutes.
– M. Prokhorov
Nov 8 at 13:24

Agree with @M.Prokhorov . Your problem is not in the domain of the Scheduling infrastructure that Spring provides. You would probably have to build your own logic. Maybe your own custom logic (or reuse lower level of task management infrastructure from Java itself or Spring) to submit tasks to an intermediate processor. That intermediate processor would keep track of timestamps, and would coordinate a proper delay between consecutive submissions.
– Arun Patra
Nov 9 at 6:04

add a comment |

up vote
0
down vote

You could do the following.

Configure the TaskExecutor. You have already done this.

Write a method that you want to invoke in a scheduled fashion and annotate with @Scheduled.
```
 @Scheduled(fixedDelay = 3600000)

 void doSomething() {

 }
```

The code above will trigger every 1 hour.

answered Nov 8 at 12:53

Arun Patra

43328

I have this setup already. My question is how to limit the API call which is currently fired about 100 times in parallel every time the job runs.
– jusermar10
Nov 8 at 13:13

This will always throttle the task, even if there were no tasks run for the past 10 minutes.
– M. Prokhorov
Nov 8 at 13:24

Agree with @M.Prokhorov . Your problem is not in the domain of the Scheduling infrastructure that Spring provides. You would probably have to build your own logic. Maybe your own custom logic (or reuse lower level of task management infrastructure from Java itself or Spring) to submit tasks to an intermediate processor. That intermediate processor would keep track of timestamps, and would coordinate a proper delay between consecutive submissions.
– Arun Patra
Nov 9 at 6:04

add a comment |

up vote
0
down vote

You could do the following.

Configure the TaskExecutor. You have already done this.

Write a method that you want to invoke in a scheduled fashion and annotate with @Scheduled.
```
 @Scheduled(fixedDelay = 3600000)

 void doSomething() {

 }
```

The code above will trigger every 1 hour.

answered Nov 8 at 12:53

Arun Patra

43328

You could do the following.

Configure the TaskExecutor. You have already done this.

Write a method that you want to invoke in a scheduled fashion and annotate with @Scheduled.
```
 @Scheduled(fixedDelay = 3600000)

 void doSomething() {

 }
```

The code above will trigger every 1 hour.

answered Nov 8 at 12:53

Arun Patra

43328

answered Nov 8 at 12:53

Arun Patra

43328

answered Nov 8 at 12:53

Arun Patra

43328

answered Nov 8 at 12:53

Arun Patra

43328

I have this setup already. My question is how to limit the API call which is currently fired about 100 times in parallel every time the job runs.
– jusermar10
Nov 8 at 13:13

This will always throttle the task, even if there were no tasks run for the past 10 minutes.
– M. Prokhorov
Nov 8 at 13:24

Agree with @M.Prokhorov . Your problem is not in the domain of the Scheduling infrastructure that Spring provides. You would probably have to build your own logic. Maybe your own custom logic (or reuse lower level of task management infrastructure from Java itself or Spring) to submit tasks to an intermediate processor. That intermediate processor would keep track of timestamps, and would coordinate a proper delay between consecutive submissions.
– Arun Patra
Nov 9 at 6:04

add a comment |

I have this setup already. My question is how to limit the API call which is currently fired about 100 times in parallel every time the job runs.
– jusermar10
Nov 8 at 13:13

This will always throttle the task, even if there were no tasks run for the past 10 minutes.
– M. Prokhorov
Nov 8 at 13:24

Agree with @M.Prokhorov . Your problem is not in the domain of the Scheduling infrastructure that Spring provides. You would probably have to build your own logic. Maybe your own custom logic (or reuse lower level of task management infrastructure from Java itself or Spring) to submit tasks to an intermediate processor. That intermediate processor would keep track of timestamps, and would coordinate a proper delay between consecutive submissions.
– Arun Patra
Nov 9 at 6:04

I have this setup already. My question is how to limit the API call which is currently fired about 100 times in parallel every time the job runs.
– jusermar10
Nov 8 at 13:13

This will always throttle the task, even if there were no tasks run for the past 10 minutes.
– M. Prokhorov
Nov 8 at 13:24

Agree with @M.Prokhorov . Your problem is not in the domain of the Scheduling infrastructure that Spring provides. You would probably have to build your own logic. Maybe your own custom logic (or reuse lower level of task management infrastructure from Java itself or Spring) to submit tasks to an intermediate processor. That intermediate processor would keep track of timestamps, and would coordinate a proper delay between consecutive submissions.
– Arun Patra
Nov 9 at 6:04

add a comment |

draft saved

draft discarded

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Name

This page is only for reference, If you need detailed information, please check here

搜尋此網誌

Xtykutl