Spring Kafka producer requires restart











up vote
0
down vote

favorite












We have an issue where a Kafka partition outage (just 3 minutes) caused our producer to time out. We are producing with a 30 second timeout with 30 retries. After the partition outage ended, our Spring Kafka service producing to the topic required a service restart to start working. We are trying to understand a configuration missing or what issue did not allow our service threads to automatically resume the ability to publish to the topic without a service restart.



Below are details on how our producers look like. If you guys see something that you think we are doing wrong or can be done better, that will be helpful.



We are using kafka version 0.10



And the only custom config we are using is – 



packEvent.producer.retries=30

packEvent.producer.acksConfig=1


Any thoughts on what was missing to not gracefully recover?






apache-kafka spring-kafka







share|improve this question




















  • 1




    I suggest you turn on DEBUG logging for org.apache.kafka; it that doesn't help; post the logs someplace like pastebin/github gist. You should also edit your question to show exactly how you are using the peoducer.
    – Gary Russell
    Nov 8 at 21:32















up vote
0
down vote

favorite












We have an issue where a Kafka partition outage (just 3 minutes) caused our producer to time out. We are producing with a 30 second timeout with 30 retries. After the partition outage ended, our Spring Kafka service producing to the topic required a service restart to start working. We are trying to understand a configuration missing or what issue did not allow our service threads to automatically resume the ability to publish to the topic without a service restart.



Below are details on how our producers look like. If you guys see something that you think we are doing wrong or can be done better, that will be helpful.



We are using kafka version 0.10



And the only custom config we are using is – 



packEvent.producer.retries=30

packEvent.producer.acksConfig=1


Any thoughts on what was missing to not gracefully recover?






apache-kafka spring-kafka







share|improve this question




















  • 1




    I suggest you turn on DEBUG logging for org.apache.kafka; it that doesn't help; post the logs someplace like pastebin/github gist. You should also edit your question to show exactly how you are using the peoducer.
    – Gary Russell
    Nov 8 at 21:32













up vote
0
down vote

favorite









up vote
0
down vote

favorite











We have an issue where a Kafka partition outage (just 3 minutes) caused our producer to time out. We are producing with a 30 second timeout with 30 retries. After the partition outage ended, our Spring Kafka service producing to the topic required a service restart to start working. We are trying to understand a configuration missing or what issue did not allow our service threads to automatically resume the ability to publish to the topic without a service restart.



Below are details on how our producers look like. If you guys see something that you think we are doing wrong or can be done better, that will be helpful.



We are using kafka version 0.10



And the only custom config we are using is – 



packEvent.producer.retries=30

packEvent.producer.acksConfig=1


Any thoughts on what was missing to not gracefully recover?






apache-kafka spring-kafka







share|improve this question















We have an issue where a Kafka partition outage (just 3 minutes) caused our producer to time out. We are producing with a 30 second timeout with 30 retries. After the partition outage ended, our Spring Kafka service producing to the topic required a service restart to start working. We are trying to understand a configuration missing or what issue did not allow our service threads to automatically resume the ability to publish to the topic without a service restart.



Below are details on how our producers look like. If you guys see something that you think we are doing wrong or can be done better, that will be helpful.



We are using kafka version 0.10



And the only custom config we are using is – 



packEvent.producer.retries=30

packEvent.producer.acksConfig=1


Any thoughts on what was missing to not gracefully recover?






apache-kafka spring-kafka




apache-kafka spring-kafka






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Nov 9 at 6:36









Giorgos Myrianthous

3,61121233




3,61121233










asked Nov 8 at 19:30









Kris

1




1








  • 1




    I suggest you turn on DEBUG logging for org.apache.kafka; it that doesn't help; post the logs someplace like pastebin/github gist. You should also edit your question to show exactly how you are using the peoducer.
    – Gary Russell
    Nov 8 at 21:32














  • 1




    I suggest you turn on DEBUG logging for org.apache.kafka; it that doesn't help; post the logs someplace like pastebin/github gist. You should also edit your question to show exactly how you are using the peoducer.
    – Gary Russell
    Nov 8 at 21:32








1




1




I suggest you turn on DEBUG logging for org.apache.kafka; it that doesn't help; post the logs someplace like pastebin/github gist. You should also edit your question to show exactly how you are using the peoducer.
– Gary Russell
Nov 8 at 21:32




I suggest you turn on DEBUG logging for org.apache.kafka; it that doesn't help; post the logs someplace like pastebin/github gist. You should also edit your question to show exactly how you are using the peoducer.
– Gary Russell
Nov 8 at 21:32

















active

oldest

votes











Your Answer






StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});


}
});














 

draft saved


draft discarded


















StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53214861%2fspring-kafka-producer-requires-restart%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown






























active

oldest

votes













active

oldest

votes









active

oldest

votes






active

oldest

votes
















 

draft saved


draft discarded



















































 


draft saved


draft discarded














StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53214861%2fspring-kafka-producer-requires-restart%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown







-

Popular posts from this blog

Schultheiß

Image
Der Titel dieses Artikels ist mehrdeutig. Weitere Bedeutungen sind unter Schultheiß (Begriffsklärung) aufgeführt. Ein Schultheiß. Holzschnitt von Peter Flötner (16. Jahrhundert) Der Schultheiß oder Schuldheiß (von althochdeutsch  sculdheizo ‚Leistung Befehlender‘, vgl. mittelniederdeutsch  schult(h)ēte , latinisiert (mittellat.) sculte(t)us , schwäbisch heute noch Schultes für „Bürgermeister“) bezeichnet einen in vielen westgermanischen Rechtsordnungen vorgesehenen Beamten, der Schuld heischt : Er hatte im Auftrag seines Herren (Landesherrn, Stadtherrn, Grundherrn) die Mitglieder einer Gemeinde zur Leistung ihrer Schuldigkeit anzuhalten, also Abgaben einzuziehen oder für das Beachten anderer Verpflichtungen Sorge zu tragen. Sprachliche Varianten des Schultheißen sind Schulte , Schultes oder Schulze . Früher wurde zwischen dem Stadtschulzen und dem Dorfschulzen unterschieden. In der städtischen Gerichts- und Gemeindeverfassung war er ein vom städtischen Rat oder vom L
Read more

Liste der Kulturdenkmale in Wilsdruff

Image
Wappen von Wilsdruff Die Liste der Kulturdenkmale in Wilsdruff enthält die Kulturdenkmale in Wilsdruff. Die Anmerkungen sind zu beachten. Diese Liste ist eine Teilliste der Liste der Kulturdenkmale im Landkreis Sächsische Schweiz-Osterzgebirge. Diese Liste ist eine Teilliste der Liste der Kulturdenkmale in Sachsen. Inhaltsverzeichnis 1 Legende 2 Wilsdruff 3 Birkenhain 4 Blankenstein 5 Braunsdorf 6 Grumbach 7 Grund 8 Helbigsdorf 9 Herzogswalde 10 Kaufbach 11 Kesselsdorf 12 Kleinopitz 13 Limbach 14 Mohorn 15 Oberhermsdorf 16 Anmerkungen 17 Ausführliche Denkmaltexte 18 Quellen 19 Weblinks Legende | Bild: zeigt ein Bild des Kulturdenkmals und gegebenenfalls einen Link zu weiteren Fotos des Kulturdenkmals Bezeichnung: Name, Bezeichnung oder die Art des Kulturdenkmals Lage: Straßenname und wenn vorhanden Hausnummer des Kulturdenkmals; Grundsortierung der Liste erfolgt nach dieser A
Read more

Android Play Services Check

Image
up vote 0 down vote favorite Im facing a problem currently im making my app to run on t.v its working fine on mobiles. but the issue is if i run my app on t.v emulator splash screen appears then the app crashes log shows this error. No acceptable module found. Local version is 0 and remote version is 0 i googled this error and came with a solution that my emulator should have latest google play services installed in it.. but in my t.v emulator there is no play store so i cant update my play services to latest version. so is there any way to make a check that if play services version is less then this so run the app instead of crashing, and if the play services version is latest also run the aap... here is my code where im getting the play services version.. int playServicesAvailable = GoogleApiAvailability.g
Read more