S rostouc\u00ed schopnost\u00ed AI agent\u016f pou\u017e\u00edvat termin\u00e1l, internet, programovac\u00ed n\u00e1stroje a vzd\u00e1len\u00e9 slu\u017eby za\u010d\u00edn\u00e1 b\u00fdt st\u00e1le d\u016fle\u017eit\u011bj\u0161\u00ed nov\u00fd typ bezpe\u010dnostn\u00edho testov\u00e1n\u00ed. Jednou ze spole\u010dnost\u00ed zam\u011b\u0159en\u00fdch pr\u00e1v\u011b na tuto oblast je Irregular.<\/p>\n\n\n\n
Firma testuje schopnosti frontier model\u016f v realistick\u00fdch sc\u00e9n\u00e1\u0159\u00edch kybernetick\u00e9 bezpe\u010dnosti a zkoum\u00e1, jak se syst\u00e9my chovaj\u00ed p\u0159i \u00faloh\u00e1ch zahrnuj\u00edc\u00edch hled\u00e1n\u00ed zranitelnost\u00ed, exploitaci, pohyb mezi syst\u00e9my nebo obch\u00e1zen\u00ed bezpe\u010dnostn\u00edch omezen\u00ed.<\/p>\n\n\n\n
Pozornost p\u0159it\u00e1hl tak\u00e9 experiment, p\u0159i kter\u00e9m AI agent v testovac\u00edm prost\u0159ed\u00ed nalezl cestu mimo p\u016fvodn\u011b zam\u00fd\u0161len\u00e9 omezen\u00ed a kompromitoval infrastrukturu souvisej\u00edc\u00ed s Hugging Face. Model dostal c\u00edl, nalezl neo\u010dek\u00e1vanou cestu k jeho spln\u011bn\u00ed a vyu\u017eil slabiny prost\u0159ed\u00ed, kter\u00e9 m\u011blo jeho mo\u017enosti omezovat. Pr\u00e1v\u011b skute\u010dnost, \u017ee to dok\u00e1zal bez explicitn\u00edho naprogramov\u00e1n\u00ed konkr\u00e9tn\u00edho \u00fatoku, je pro bezpe\u010dnostn\u00ed v\u00fdzkum zaj\u00edmav\u00e1.<\/p>\n\n\n\n
Tady podle m\u011b za\u010d\u00edn\u00e1 jedna z nejd\u016fle\u017eit\u011bj\u0161\u00edch bezpe\u010dnostn\u00edch debat cel\u00e9 agentn\u00ed \u00e9ry. Dokud AI pouze generovala text v chatovac\u00edm okn\u011b, jej\u00ed chyba v\u011bt\u0161inou skon\u010dila nesmyslnou odpov\u011bd\u00ed. A nen\u00ed v\u016fbec nutn\u00e9, aby m\u011bl model n\u011bjak\u00fd \u201ezl\u00fd \u00famysl\u201c.<\/p>\n\n\n\n
Sta\u010d\u00ed kombinace agresivn\u011b optimalizovan\u00e9ho c\u00edle, nedostate\u010dn\u011b definovan\u00fdch omezen\u00ed a zraniteln\u00e9ho prost\u0159ed\u00ed. Proto bude red teaming a testov\u00e1n\u00ed agentn\u00edch syst\u00e9m\u016f v sandboxech pravd\u011bpodobn\u011b stejn\u011b d\u016fle\u017eit\u00e9 jako samotn\u00fd v\u00fdvoj jejich schopnost\u00ed.<\/p>\n","protected":false},"excerpt":{"rendered":"
S rostouc\u00ed schopnost\u00ed AI agent\u016f pou\u017e\u00edvat termin\u00e1l, internet, programovac\u00ed n\u00e1stroje a vzd\u00e1len\u00e9 slu\u017eby za\u010d\u00edn\u00e1 b\u00fdt st\u00e1le d\u016fle\u017eit\u011bj\u0161\u00ed nov\u00fd typ bezpe\u010dnostn\u00edho testov\u00e1n\u00ed. Jednou ze spole\u010dnost\u00ed zam\u011b\u0159en\u00fdch pr\u00e1v\u011b na tuto oblast je Irregular. Firma testuje schopnosti frontier model\u016f v realistick\u00fdch sc\u00e9n\u00e1\u0159\u00edch kybernetick\u00e9 bezpe\u010dnosti a zkoum\u00e1, jak se syst\u00e9my chovaj\u00ed p\u0159i \u00faloh\u00e1ch zahrnuj\u00edc\u00edch hled\u00e1n\u00ed zranitelnost\u00ed, exploitaci, pohyb mezi […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-231","post","type-post","status-publish","format-standard","hentry","category-nezarazene"],"_links":{"self":[{"href":"https:\/\/www.martinpavlicek.cz\/blog\/wp-json\/wp\/v2\/posts\/231","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.martinpavlicek.cz\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.martinpavlicek.cz\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.martinpavlicek.cz\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.martinpavlicek.cz\/blog\/wp-json\/wp\/v2\/comments?post=231"}],"version-history":[{"count":1,"href":"https:\/\/www.martinpavlicek.cz\/blog\/wp-json\/wp\/v2\/posts\/231\/revisions"}],"predecessor-version":[{"id":232,"href":"https:\/\/www.martinpavlicek.cz\/blog\/wp-json\/wp\/v2\/posts\/231\/revisions\/232"}],"wp:attachment":[{"href":"https:\/\/www.martinpavlicek.cz\/blog\/wp-json\/wp\/v2\/media?parent=231"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.martinpavlicek.cz\/blog\/wp-json\/wp\/v2\/categories?post=231"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.martinpavlicek.cz\/blog\/wp-json\/wp\/v2\/tags?post=231"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}